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MEMORANDUM  FOR 

DIRECTOR,  PROGRAM  ANALYSIS  AND  EVALUATION  (DPAE),  OFFICE  OF  THE 
CHIEF  OF  STAFF,  U.S.  ARMY,  WASHINGTON,  DC  20310-0200 
TECHNICAL  ADVISOR  TO  THE  DEPUTY  CHIEF  OF  STAFF  FOR  OPERATIONS 
AND  PLANS,  WASHINGTON,  DC  20310-0410 

SUBJECT;  Army  Program  Value  Added  Analysis  90-97  (VAA  90-97) 
Study 


1.  Reference  memorandum.  Office  of  the  Chief  of  Staff,  U.S. 
Army,  10  May  1989,  subject:  Army  Program  Value  Added  Analysis 
90-97  (VAA  90-97). 

2.  Reference  memorandum  requested  that  the  U.S.  Army  Concepts 
Analysis  Agency  (CAA)  conduct  a  study  to  provide  the  Director, 
Program  Analysis  and  Evaluation  and  the  Office  of  the  Deputy 
Chief  of  Staff  for  Operations  and  Plans  (ODCSOPS)  an  analytical 
methodology  and  decision  support  system  for  the  development  of 
a  balanced  and  effective  Army  Program. 

3.  This  final  report  documents  the  results  of  our  analysis  and 
incorporates  your  comments  on  the  interim  report.  The  Value 
Added  Analysis  (VAA)  methodology  is  a  decision  support  system 
that  will  assist  decisionmakers  in  evaluating  and  prioritizing 
competing  alternatives  in  the  POM  building  process.  The  Value 
Added  Analysis  concept  uses  a  family  of  models  to  measure  an 
alternative's  explicit  (objective)  contribution  to  the  prog~am 
as  an  incremental  or  decremental  change  to  the  current  pm/ram 
base.  A  hierarchical  assessment  framework  is  used  to  develop 
an  alternative's  scores.  This  assessment  framework  is  j^ed  to 
evaluate  changes  against  the  current  program  base  as  '■iie 
consequences  of  program  alternatives  are  considered.  Value 
Added  Analysis  results  in  measuring  an  alternative's  relative 
value  in  the  context  of  a  larger  value  system.  This  relative 
value  is  either  used  directly  by  decisionmakers  or  is  fed  into 
a  mathematical  optimization  model  which  simultaneously 
determines  an  alternative's  cost-benefit  and  conducts  a 

«  tradeoff  between  alternatives. 
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SUMMARY 

CAA-SR-91-9 

THE  REASON  FOR  PERFORMING  THE  STUDY  was  to  provide  the  Director  for 
Program  Analysis  and  Evaluation,  and  the  Deputy  Chief  of  Staff  for  Operations 
and  Plans  (DCSOPS)  an  analytical  methodology  and  capability  to  support  the 
development  of  a  balanced  and  effective  Army  Program. 

THE  STUDY  SPONSORS  are  the  Director  for  Program  Analysis  and  Evaluation 
(DPAE),  Office  of  the  Chief  of  Staff,  Army,  and  the  Technical  Advisor,  Office 
of  the  Deputy  Chief  of  Staff  for  Operations  and  Plans  (ODCSOPS),  Head¬ 
quarters,  Department  of  the  Army  (HQDA). 

THE  STUDY  OBJECTIVES  were  to: 

(1)  Formulate  an  analytic  method  for  estimating  the  marginal  value  of 
competing  management  decision  packages  (MDEP)  to  the  Army.  The  methodology 
will  provide  a  common  understandable  basis  for  the  analysis  of  affordability 
issues. 

(2)  Identify  or  develop  prototype  models  that  support  the  Value  Added 
Analysis  methodology  and  provide  management  tools  for  the  analytic  method 
developed, 

(3)  Establish  within  DPAE  and  ODCSOPS  an  in-house  capability  to  conduct 
program  issue  tradeoffs. 

(4)  Conduct  a  proof  of  concept  analysis  during  the  building  of  the  92-97 
Program  Objective  Memorandum  (POM). 

THE  SCOPE  OF  THE  STUDY  included  the  research,  development,  and  acquisition 
(RDA)  appropriations  for  selected  items  of  equipment  in  fiscal  year  (FY)  1999 
and  FY  2004. 

THE  MAIN  ASSUMPTION  of  this  study  is  that  HQDA  needs  a  relatively  quick 
method  for  conducting  program  tradeoffs  which  has  sound  analytical 
underpinnings. 

THE  BASIC  APPROACH  of  this  study  was  to: 

(1)  Identify  and  develop  an  analytical  approach  to  evaluate  program  issue 
tradeoffs. 

(2)  Develop  a  capability  for  implementing  the  methodology  to  include 
software  modules  where  appropriate. 


V 


(3)  Deaonstrate  the  methodology  and  capability  by  using  Issues  from  the 
90-97  POM  Issue  cycle. 

THE  PRINCIPAL  FINDINGS  of  the  study  were: 

(1)  Two  generalized  categories  of  measures  of  effectiveness  (explicit  and 
Implicit)  were  determined  to  be  Important  in  judging  relative  value. 

(2)  A  linear  combination  of  value  components  appears  to  be  useful  in 
creating  a  single  measure  of  an  investment's  marginal  value. 

(3)  VAA  uses  a  system  of  judgment  weights  which  measures  an  investment's 
value  In  a  dual  context. 

(4)  The  VAA  methodology  as  demonstrated  holds  promise  as  the  decision 
analysis  framework  that  should  be  used  for  conducting  Army  program  tradeoff 
analyses.  Further  work  In  the  areas  of  quick  turnaround  combat  modeling, 
dynamic  costing,  and  data  collection  need  to  be  conducted. 

(5)  A  highly  aggregated  combat  model  can  provide  useful  Insights  for 
answering  macro-level  program  tradeoff  questions  and,  to  a  more  limited 
degree,  is  capable  of  providing  specific  program  guidance. 

(6)  In  order  for  aggregated  combat  models  to  be  useful,  they  must  be 
calibrated  to  more  detailed  models  and  data. 

(7)  HQOA  has  a  perspective  which  does  not  Include  the  Training  and 
Doctrine  Command's  (TRAOOC)  blueprint  of  the  battlefield  per  se.  Their 
headquarters  is  organized  along  the  lines  of  Planning,  Programming, 

Budgeting,  and  Estimation  System  (PPBES)  functions,  and,  although  the 
headquarters  has  some  overlap,  the  specific  thrust  is  the  generation  of 
forces  through  planning  and  programming. 

(8)  The  cost-benefit  analysis  In  the  form  of  the  optimization  model 
requires  further  research  Into  the  use  of  dynamic  costing  of  the  procurement 
costs. 

THE  STUDY  EFFORT  was  directed  by  LTC  James  A.  Richmann,  Force  Systems 
Directorate,  US  Army  Concepts  Analysis  Agency  (CAA). 

COMMENTS  AND  QUESTIONS  may  be  sent  to  the  Director,  US  Army  Concepts 
Analysis  Agency,  AHN:  CSCA-FSR,  8120  Woodmont  Avenue,  Bethesda,  Maryland 
20814-2797. 
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CHAPTER  1 
EXECUTIVE  SUMMARY 


1—1.  PROBLEM.  The  leadership  at  Department  of  the  Army 
needs  analysis  to  support  the  development  of  a  balanced  and 
effective  Army  program  that  is  within  Department  of  Defense 
resource  guidance. 


1—2.  STUDY  PURPOSE.  The  purpose  of  the  Army  Program 
Value  Added  Analysis  (VAA)  was  to  provide  the  Director  for 
Program  Analysis  and  Evaluation  (DPAE),  and  the  Deputy  Chief 
of  Staff  for  Operations  and  Plans  (DCSOPS)  with  an  analytical 
tradeoff  methodology  and  capability  that  would  assist  the 
development  of  a  balanced  and  effective  Army  program  through 
the  use  of  a  comprehensive  cost-benefit  analysis. 


1-3 .  BACKGROUND 


a.  Traditionally,  Army  program  development  is  accom¬ 
plished  during  the  Planning,  Programming,  Budgeting,  and 
Execution  System  (PPBES)  Program  Objective  Memorandum  (POM) 
building  process  using  some  form  of  functional  or  mission 
area  panels.  The  processes  that  evolved  do  not  include 
adequate  means  of  integrating  these  functional  subprograms 
into  a  balanced  total  Army  program.  Furthermore,  the 
traditional  POM  process  allowed  each  panel  to  use  its  own 
methods  for  prioritization.  This  lack  of  standardization 
prevents  the  senior  leadership  from  making  the  most  effective 
tradeoffs  across  functional  areas. 


b.  Virtually  all  analysis  currently  performed  in  program 
evaluation  focuses  on  defining  individual  Management  Decision 
Package  (MDEP)  issues.  When  MDEP  analysis  is  conducted,  the 
senior  Army  leadership  lacks  the  visibility  or  analysis 
necessary  to  help  identify  the  marginal  value  of  resources 
within,  or  across,  MDEPs. 


c.  Analysis  of  the  total  Army  Program  requires  an 
understanding  of  how  individual  MDEPs  contribute  to  the  Army 
mission  and  strategy  so  as  to  determine  which  MDEPs  and 
resource  levels  have  the  greatest  return  on  investment.  One 
approach  to  this  problem  is  to  estimate  the  "value  added"  by 
individual  MDEPs,  or  groups  of  MDEPs — expressed  as  program 
enhancement  packages  or  alternative  solutions--to  the  total 
Army  program  as  measured  by  their  contribution  to  Army 
objectives . 
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d.  The  Value  Added  Analysis  concept  uses  a  family  of 
models  to  measure  an  issue's  explicit  (objective)  contri¬ 
bution  to  the  program  as  an  incremental  or  decremental  change 
from  the  current  program  base.  A  pairwise  comparison 
framework  is  used  to  develop  an  issue's  implicit  (subjective) 
contribution  to  the  program  through  the  development  of 
individual  alternative’s  scores.  Saaty’s  analytical 
hierarchy  technique  is  used  to  provide  a  structure  for 
developing  weights  for  both  the  explicit  and  implicit 
measures  of  value. 


1-4.  STUDY  OBJECTIVES  were  to: 

a.  Formulate  an  analytic  method  for  estimating  the 
marginal  value  of  competing  MDEPs  to  the  Army.  The 
methodology  will  provide  a  common  understandable  basis  for 
the  analysis  of  affordability  issues. 


b.  Identify  or  develop  prototype  models  that  provide 
management  tools  and  support  the  Value  Added  Analysis 
methodology. 


c.  Establish  within  PAED  and  ODCSOPS  an  in-house 
capability  to  analyze  program  issue  tradeoffs. 


d.  Conduct  a  proof  of  concept  analysis  during  the 
building  of  the  Fiscal  Year  (FY)  92—97  POM. 


1—5.  STUDY  SCOPE.  Only  research,  development  and 
acquisition  (RDA)  appropriations  and  associated  life  cycle 
costs  for  selected  items  of  equipment  were  included. 


1-6.  STUDY  LIMITATIONS 

a.  Since  this  study  is  a  proof  of  concept  demonstration, 
the  analysis  included  only  the  RDA  appropriations  Only  RDA 
was  considered  because  of  the  need  to  limit  the  scope  of  the 
study  to  those  appropriations  with  good  data  availability  and 
issue  definition.  However,  the  final  methodology  should  be 
capable  of  examining  tradeoffs  across  the  full  range  of 
program  issues. 

b.  Because  the  modeling  and  effectiveness  data  for  the 
noncombat  issues  are  less  developed  than  the  combat  areas, 
the  study  focused  principally  on  combat  tradeoff  issues. 


c.  The  original  list  of  equipment  issues  for  this  study 
included  23  systems.  Limited  data  available  for  combat 
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modeling  did  not  permit  the  inclusion  of  more  than  12  systems 
in  the  analysis. 


d.  The  initial  data  collection  design  called  for  system 
effectiveness,  force  structure,  cost,  and  production  data  for 
FYs  1991,  1999,  and  2004-  Effectiveness  data  could  not  be 
collected  for  all  three  years  in  a  timely  manner.  Therefore, 
only  FY  1999  and  FY  2004  data  were  used. 


e.  RDA  appropriations  provide  only  22  percent  of  the  Army 
program  dollars . 


1—7.  TIMEFRAME.  This  proof  of  concept  analysis  addresses 
tradeoff  issues  during  the  period  beginning  in  FY  99  and 
ending  in  FY  04. 


1-8.  KEY  ASSUMPTIONS 

a.  The  methodology  assumes  that  all  tradeoff  issues  will 
only  be  those  on  the  margin,  and  does  not  purport  to  realign 
previous  (years  prior  to  current  POM)  MDEP  funding  levels  or 
quantities.  The  Value  Added  Analysis  will  not  be  used  to 
build  a  completely  new  program  from  a  zero  base.  All 
previous  POM  decisions  which  resulted  in  the  current  program 
would  remain  unaltered.  VAA  would  only  be  used  to  deal  with 
increments  and  decrements  starting  from  a  base  represented  by 
the  current  POM  (FY  92-97)  position. 

b.  The  Value  Added  Analysis  can  be  executable  in  a  quick 
turnaround  environment.  Therefore, the  combat  models  applied 
must  process  at  a  lower  level  of  resolution  using  data  that 
has  been  aggregated  from  more  detailed  models  or  data  bases. 


c.  VAA  must  be  compatible  with  the  manner  in  which  the 
senior  Army  leadership  makes  decisions. 


d.  The  Value  Added  Analysis  assumes  a  two-tier  decision 
process  wherein  the  senior  Army  leadership  provides  guidance 
regarding  Total  Obligation  Authority  (TOA  )  allocation  (first 
tier),  and  the  VAA  methodology  is  used  to  decide  on  tradeoff 
issues  within  this  guidance  (second  tier).  However,  the  TOA 
guidance  may  or  may  not  be  developed  using  the  Value  Added 
methodology. 


e.  The  methodology  must  allow  for  prioritizing  between 
dissimilar  program  alternatives. 
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1-9 .  STUDY  METHODOLOGY 

a.  The  Value  Added  Analysis  concept  uses  a  hierarchical 
assessment  framework  for  developing  a  single  measure  of 
benefit.  This  assessment  framework  is  used  to  portray 
existing  capability  (current  program  levels)  and  evaluates 
capability  changes  against  this  base  level  as  the  impact  of 
alternative  solutions  for  program  tradeoffs  issues  are 
considered.  The  contribution  of  a  program  alternative  is 
measured  both  explicitly  (using  combat  models  and  other 
quantitative  methods)  and  implicitly  (using  Thomas  L.  Saaty's 
pairwise  comparison  technique^).  VAA  results  in  measuring  an 
alternative's  relative  value  in  the  context  of  a  larger  value 
context.  This  relative  value  is  used  either  directly  by 
decisionmakers  or  is  input  into  a  linear  program  to  determine 
an  alternative's  cost  effectiveness,  and,  in  turn, 
acquisition  strategy 


b.  The  current  application  of  the  VAA  methodology  uses  four 
types  of  models;  combat  models,  costing  models, 
multiattribute  models,  and  linear  programing  models.  All  of 
them  use  existing,  well-accepted  operations  research 
techniques  to  eliminate  any  debate  over  those  specific 
techniques  used.  Furthermore,  by  using  techniques  which  many 
people  are  already  familiar  with,  the  learning  curve  for 
using  the  VAA  methodology  would  be  shortened.  Figure  1-1 
depicts  the  overall  Value  Added  methodology. 
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c.  The  combat  and  costing  models  are  used  to  obtain  the 
explicit  (objective)  measures  of  effectiveness  of  an 
investment  alternative.  The  implicit  measures  of  effec¬ 
tiveness  are  developed  through  the  use  of  two  surveys.  The 
first  survey  is  given  to  the  Army  leadership  as  a  means  of 
deriving  decision  weights. for  subjective  decision  criteria 
(e.g.,  Congressional  opinion).  These  criteria  are  referred 
to  as  secondary  impact  analysis  modifiers  (SIAM  factors)  and 
are  developed  using  Saaty’s  pairwise  comparison  technique. 

The  second  survey  is  given  to  subject  matter  experts,  who  are 
asked  to  value  or  score  investment  alternatives  in  light  of 
the  SIAM  factors. 


d.  The  explicit  measures  of  benefit  are  combined  with  the 
implicit  (subjective)  measures  using  a  model  called  TOPSIS 
(Technique  for  Order  Preference  by  Similarity  to  Ideal 
Solution) .  This  combined  measure  of  benefit  is  evaluated 
against  cost  using  a  multiobjective  linear  program  called 
FOMOA  (Force  Modernisation  Analyzer).  FOMOA  produces 
efficient  acquisition  strategy  based  upon  a  cost  effective 
criterion.  This  display  may  be  used  as  is  or  processed 
further  using  a  future  enhancement  which  will  allow  the 
analyst  to  distribute  equipment  or  program  items  by  unit  or 
activity. 


e.  The  previous  paragraphs  of  this  chapter  provide  a 
brief  outline  of  the  VAA  methodology.  Readers  desiring  a 
fundamental  understanding  of  this  study  and  the  VAA 
methodology  should  read  Chapters  2,  3,  10,  and  Appendix  E. 
Those  needing  a  more  detailed  explanation  should  read 
Chapters  2  through  10  and  Appendices  D  and  E.  Individuals 
wishing  to  implement  the  VAA  methodology  should  read  Chapters 
2  through  10  plus  Appendices  D  through  H. 


1-10.  ESSENTIAL  ELEMENTS  OF  ANALYSIS  (EEA) .  The 
essential  elements  of  analysis  and  basic  findings  for  each 
are  as  follows: 

a.  EEA  1:  What  measures  of  effectiveness  (MOE) 

are  appropriate  for  judging  relative  value  among  all 
MDEPs  in  the  program?  Two  generalized  categories  of 
measures  of  effectiveness  were  determined  to  be  important  in 
judging  relative  value. 


(1)  The  first  (called  explicit  effectiveness)  consists 
of  three  subdivided  elements  (combat  effectiveness,  soldier 
quality  of  life,  and  other  Army  goals  and  objectives).  They 
are  those  measures  which  can  be  directly  quantified.  These 
measures  include  such  MOE  as  force  exchange  ratios  (FER), 
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correlation  of  forces  and  means  (COFM),  day  care  days,  amount 
of  energy  savings,  etc.  These  MOE  were  selected  for  their 
ability  to  appropriately  measure  the  issue  in  its  own 
context . 

(2)  The  second  category  of  MOE  (SIAM  factors)  measures 
implicit  effectiveness  and  includes  those  measures  which  are 
inherently  qualitative,  such  as  political  risk  and  insti¬ 
tutional  stability.  These  MOE  were  selected  because  they 
express  subjective  values  which  are  best  measured  through 
expert  judgment. 

b.  EEA  2:  Should  a  linear  combination  of  value 

components  be  used  to  arrive  at  a  single  measure  of 
value  added?  If  so,  what  system  of  judgment  weights 
should  be  used? 

(1)  A  linear  combination  of  value  components  appears 
to  be  useful  in  creating  a  single  measure  of  an  issue's 
marginal  value. 

(2)  VAA  uses  a  system  of  judgment  weights  which 
measures  the  value  of  an  issue  in  a  dual  context.  The 
methodology  seeks  two  sets  of  estimates.  The  first  set  of 
estimates  is  created  by  surveying  the  Army  leadership  to 
determine  both  the  set  of  judgmental  factors  and  their 
weights  in  accordance  with  the  group's  value  set  and  world 
view.  The  second  set  of  estimates  is  created  by  scoring 
individual  issues  using  subject  matter  experts  and  the 
judgmental  factors  developed  by  surveying  the  leadership. 

Both  sets  of  surveys  appear  to  yield  consistent  and 
informative  values.  These  approaches  are  successful  in 
capturing  the  multiple  attributes  used  by  the  decisionmakers 
in  the  PPBES  programing  decision  process. 

c.  EEA  3:  What  decision  analysis  framework  should 

be  used?  The  VAA  methodology  as  demonstrated  in  Phase  I 
holds  promise  as  the  decision  analysis  freutework  that  should 
be  used  for  conducting  Anny  program  tradeoff  analyses. 

Further  work  in  the  areas  of  quick  turn-around  combat 
modeling,  dyncimic  costing,  and  data  collection  need  to  be 
conducted . 

d.  EEA  4:  Does  a  highly  aggregated  combat  model 

provide  useful  insights  for  answering  program-wide 
tradeoff  questions?  If  so,  can  such  a  model  be 
calibrated  to  a  more  detailed  model  if  necessary? 

(1)  A  highly  aggregated  combat  model  can  provide 
useful  insights  for  answering  macro-level  program  tradeoff 
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questions  and,  to  a  more  limited  degree,  is  capable  of 
providing  specific  program  guidance. 

(2)  In  order  for  aggregated  combat  models  to  be  fully 
useful,  they  must  be  calibrated.  In  order  to  calibrate  them, 
one  must  be  able  to  establish  some  benchmark  values  or 
calibration  points  using  data  or  output  from  accepted  and 
"known"  sources.  These  points  do  not  necessarily  need  to  be 
developed  from  other  models  of  higher  resolution. 

Calibration  can  be  done  using  other  models,  empirical  data, 
or  historical  data.  The  VAA  Study  attempted  to  calibrated  a 
medium  resolution  combat  model  (Corps  Battle  Analyzer 
[CORBAN])  with  a  more  detailed  medium  resolution  model 
(Vector- In-Commander  [VIC]).  This  effort  at  calibrating  one 
model  with  results  from  another  was  not  fully  successful 
because  we  did  not  have  both  models  available  to  use  (VIC  was 
not  run  at  CAA) .  The  team  was  only  successful  at  matching  the 
tempo  of  the  battles  in  CORBAN  in  relation  to  the  6.3 
scenario  in  VIC.  Further  work  on  calibration  should  be 
conducted . 


e.  EEA  5:  What  Blueprint  (of  the  battlefield) 

options  provide  the  most  useful  framework  for  the 
analysis  of  change  to  the  Army's  materiel,  force 
structure,  etc?  TRADOC's  Blueprint  of  the  Battlefield, 
which  this  EAA  was  directed  towards,  appears  not  to  be  useful 
within  the  context  of  VAA.  The  blueprint  concept  is  oriented 
by  its  very  nature  toward  a  requirements  perspective  and  does 
not  necessary  address  the  issues  of  affordability  or 
duplication  of  capability.  The  AHP  process  appears  to  hold 
the  greatest  possibilities  in  this  regard. 

1-11.  OTHER  KEY  FINDINGS 

a.  The  Value  Added  methodology  is  data  intensive. 

Accuracy  and  timeliness  of  input  data,  especially  production 
data,  is  of  crucial  importance  in  making  the  value  added 
process  operational. 


b.  The  technique  used  to  survey  the  Army  leadership 
allowed  decision  making  attitudes  and  behaviors  to  be 
accurately  reflected. 

c.  The  Value  Added  methodology  will  "buy"  the  most  cost 
effective  systems  subject  to  constraints. However,  in  using  a 
fixed  average  unit  cost  (AUC)  in  the  optimization  model  the 
current  configuration  creates  a  shortcoming  which  needs  to  be 
addressed.  The  danger  exists  that  in  recomputing  AUC  after 
the  optimization  has  been  completed  the  total  cost  of  the 
quantities  recommended  could  exceed  the  TOA  given.  VAA  Phase 
II  must  address  dyneimic  costing  within  the  optimization  so  as 
to  recompute  the  procurement  cost  as  part  of  the  optimization 
process . 
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d.  The  study  developed  a  modular  "living"  methodology 
with  a  high  degree  of  flexibility  which  can  be  used  with  a 
variety  of  techniques  and  models. 


e.  The  methodology  provides  a  standardized  quick  reaction 
approach  that  uses  operations  research/ systems  analysis 
(ORSA)  techniques.  It  evaluates  program  tradeoff  issues  by 
functional  area  experts  and  decisionmakers.  ORSA  expertise 
is  required  during  the  preparatory  analysis,  consisting  of 
the  combat  modeling,  costing,  and  surveying.  The  functional 
analysis  which  would  take  place  during  the  POM  building  and 
defense,  consisting  of  the  actual  tradeoffs,  does  not  require 
extensive  ORSA  expertise  when  using  the  VAA  methodology. 


f.  The  scope  of  Value  Added  Analysis  needs  to  be  expanded 
beyond  the  RDA  appropriations,  since  these  contain  only  about 
22  percent  of  the  Army  program  dollars. 
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CHAPTER  2 
INTRODUCTION 


2-1.  OVERVIEW.  The  purpose  of  this  chapter  is  to  provide  a 
discussion  of  the  background  which  influenced  the  development  of  the 
Value  Added  methodology.  Chapter  3  provides  an  overview  of  the 
methodology  without  respect  to  those  demonstration  case  studies  that 
were  actually  conducted  during  this  study.  The  case  studies  and 
their  results  are  presented  in  Appendix  E,  which  is  published 
separately.  Subsequent  chapters  (4-9)  will  present  a  more  detailed 
discussion  of  the  methodology  using  an  excunple  tradeoff  issue  for 
armor/ antiarmor  systems.  It  should  be  noted  from  the  study  directive 
that  VAA  was  to  be  limited  to  development  of  a  methodology  and 
prototype  models.  By  the  very  nature  of  this  study,  the  team 
understood  that  subsequent  efforts  aimed  at  refinement  would  be 
necessary.  Therefore,  a  section  of  the  svimmary  chapter  will 
identify,  where  possible,  those  areas  requiring  further  work. 

Chapter  2  is  structured  as  follows: 

a  .  Problem  Background 

b .  Scope/Limitations/Timeframe 

c  .  Key  Methodological  Assumptions 

d  .  Influences  on  the  Approach 

e .  Summary 

2-2.  PROBLEM  BACKGROUND 

a.  The  growing  federal  budget  deficit  and  the  breakout  of  peace 
in  Europe  have  put  great  pressure  on  all  the  armed  services  to  re¬ 
evaluate  and  restructure  their  budget  and  programs .  This  is 
especially  true  for  the  Army  because  of  the  changing  perception  of 
the  threat  in  Western  Europe.  There  is  more  need  now  for  cost- 
benefit  analysis  than  ever  before.  This  analysis  must  be  conducted 
in  support  of  developing  a  balanced  and  effective  Army  progreim  within 
Department  of  Defense  resource  guidance.  Traditionally,  the  Army  has 
used  functional  area  panels  to  build  its  POM.  The  processes  that 
have  evolved  do  not  include  adequate  means  of  integrating  and 
balancing  the  functional  programs  into  the  Total  Army  Program.  The 
Value  Added  Analysis  has  been  conceived  as  a  means  of  assisting  in 
accomplishing  this  integration  and  balancing. 

b.  The  environment  in  which  Value  Added  Analysis  must  be 
conducted  is  characterized  by  minimal  time  for  analysis,  ever- 
changing  assumptions,  often  incomplete  data,  and  increasing  emphasis 
on  verifiable  results.  Traditional  staff  processes  and  supporting 
operations  research  studies  often  do  not  meet  this  challenge.  This 
failure  occurs  because  traditional  approaches  require  a  long  time  and 
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highly  trained  personnel  to  produce  high  quality,  fully  integrated, 
and  very  detailed  analysis. 

c.  The  requirement  for  a  Value  Added  Analysis  (VAA)  methodology 
was  conceived  by  the  Director,  PAED  as  a  means  of  conducting  program 
tradeoff  analyses.  In  early  1989,  the  Director,  PAED,  asked  the 
Technical  Advisor,  Deputy  Chief  of  Staff  for  Operations  and  Plans,  to 
jointly  sponsor  a  study  to  be  conducted  by  the  US  Army  Concepts 
Analysis  Agency  (CAA)  to  develop  the  methodology. 

2-2.  SCOPE/LIMITATIONS/TIMEFRAME 

a.  The  VAA  Study  was  designed  as  a  proof  of  concept  to  include  the 
building  of  a  prototype  decision  support  system.  Because  the  work 
was  developmental,  we  limited  the  scope  to  weapon  system  tradeoffs 
and  did  not  attempt  to  address  force  structure,  personnel, 
infrastructure,  or  other  issues.  The  methodology  is,  however, 
designed  to  be  generic  and  used  across  all  the  functional  areas  of 
the  Army  Program. 

b.  Because  the  first  objective  of  this  study  was  to  develop  a 
methodology,  it  was  decided  to  limit  the  program  alternative 
tradeoffs  to  weapon  system  mix  issues.  This  limitation  by  its  very 
nature  focused  the  study  on  the  RDA  appropriation.  Research, 
development,  testing,  and  evaluation  (RDTE)  and  other  equipment- 
related  appropriations  such  as  operation  and  maintenance.  Army  (OMA) 
(for  operation  and  sustainment  (O&S))  were  either  not  considered  or 
used  in  a  very  limited  manner.  However,  it  is  clear  that  the 
methodology  must  be  capable  of  examining  tradeoffs  across  the  full 
range  of  program  issues  and  therefore  across  all  appropriations. 

c.  The  VAA  study  directive  required  that  the  methodology  consider 
both  combat  and  noncombat  measures  of  effectiveness.  However, 
principally  because  of  limited  time,  limited  data,  and  inadequate 
noncombat  models,  the  study  focused  primarily  on  combat  related 
tradeoff  issues. 

d.  The  VAA  study  directive  required  that  the  methodology 
consider  implementation  in  an  IBM  PC-compatible  computer 
environment.  This  requirement  was  identified  because  PAED  and 
other  major  Army  Staff  (ARSTAF)  agencies  conduct  their  work  in 
the  IBM  environment. 

e.  The  following  terms  of  reference  were  used  in  developing  the 
VAA  Study. 

(1)  FY  90/91  amended  budget. 

(2)  FY  94  program  force  for  the  base. 

(3)  NATO  Central  Europe  scenario. 

(4)  Conventional  conflict  (no  chemical  or  nuclear). 
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f.  As  part  of  the  study  directive,  TRAC  (TRADOC  Analysis 
Command),  Fort  Leavenworth,  was  tasked  to  assist  CAA  in  conducting 
the  corps-level  combat  simulations.  The  VAA  effort  was  a  significant 
addition  to  TRADOC 's  Army  Regulation  (AR)  5-25  FY  90  study  program. 
Because  VAA  represented  an  unplanned  additional  requirement,  TRADOC 
requested  that  VAA  be  linked  to  the  ODCSOPS  Army  Requirements  and 
Capabilities  Study  (ARCPS).  One  outgrowth  of  this  linkage  was  the 
selection  of  timeframes  for  the  VAA  Study.  ARCPS  required  a  set  of 
dates  that  allowed  for  looking  at  program  growth  starting  at  the  end 
of  the  budget  year,  going  through  the  POM  years  and  ending  7  years 
into  the  extended  planning  years.  This  requirement  resulted  in  the 
following  years  being  selected  for  data  collection  and  combat 
simulations — FY  1991,  FY  1999,  and  FY  2004.  However,  because  of  data 
limitations,  FY  1991  was  deleted  from  the  study. 

2-3.  KEY  METHODOLOGICAL  REQUIREMENTS 

a.  The  analysis  will  expand  to  other  appropriations  and 
issues  as  the  methodology  matures . 

b.  The  decision  support  system  and  related  models  must  be 
capable  of  operating  in  a  quick  turnaround  environment,  defined 
as  1  week  or  less. 

c.  The  above  requirement  led  to  the . following  additional 
requirements : 

(1)  Combat  models  for  use  in  Value  Added  Analysis  must 
process  aggregated  data  and  be  calibrated  to  higher  resolution 
models,  empirical  data,  or  historical  data. 

(2)  Data  manipulation  must  be  easy  and  very  quick. 

(3)  The  decision  support  system  must  tap  major 
authoritative  Army  data  bases  such  as  the  Total  Army  Equipment 
Distribution  Program  (TAEDP),  Force  Accounting  System  (FAS),  and 
line  item  number  (LIN)  Price. 

d.  The  methodology  must  accommodate  the  manner  in  which  the 
senior  Army  leadership  makes  decisions. 

e.  The  methodology  must  allow  for  prioritizing  between 
dissimilar  program  alternatives  (weapon  systems  versus  training 
versus  family  member  support). 

f.  Input  data  will  come  from  a  wide  variety  of  sources,  may 
prove  to  be  difficult  to  obtain,  and  must  be  scrubbed  for 
verification/validation. 

2-4.  INFLUENCES  TO  THE  APPROACH 

a.  A  Definition  of  Value 
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(1)  As  one  thinks  about  the  term  "value,"  it  becomes  clear 
that  value  is  always  contextual.  In  making  judgments  about  the  value 
of  something,  we  normally  measure  that  item  against  a  standard. 

These  standards  may  be  developed  from  a  complex  system  of  comparative 
examples,  such  as  the  value  system  we  help  imbue  in  our  children.  Or 
the  standard  may  be  as  simple  as  a  single  measure  such  as,  "How  much 
does  my  house  cost  (in  dollars)  in  relation  to  others  on  the  market?" 
In  both  cases,  criterion  and  means  to  measure  an  alternative  against 
the  criterion  were  established.  Value  is  "in  the  eye  of  the 
beholder,"  and  therefore  is  inextricably  linked  to  the  decision¬ 
makers'  value  system.  This  is  also  true  for  the  Value  Added 
Analysis.  The  process  of  determining  value  can  be  viewed  as  the 
defining  of  a  context,  the  establishment  of  criterion  and  associated 
standard,  and  the  selection  of  a  means  to  measure  choices 
(alternatives)  against  the  criterion  and  standard. 

(2)  For  the  Value  Added  Analysis  methodology,  the  idea  of 
defining  a  context  is  the  most  important  concept  to  understand.  It  is 
from  coiitext  that  the  whole  evaluation  process  obtains  its  meaning. 
For  example,  the  demonstration  case  study  uses  a  context  of  a  NATO 
Central  Europe  battle  (pre-CFE).  However,  it  is  just  as  valid  to  use 
a  Southwest  Asia  or  Northeast  Asia  scenario.  In  fact,  one  might  want 
to  look  at  all  three  scenarios  to  gain  insights  on  how  the  value  of  a 
particular  set  of  alternatives  changes  from  theater  to  theater. 
Furthermore,  this  theater  context  may  be  modified  by  the  fact  that 
the  Federal  budget,  and  therefore  the  Army  budget,  is  decreasing. 

VAA,  as  envisioned  by  the  sponsors  of  this  study,  embraces  the- 
context  of  the  importance  of  judging  program  alternatives  against  the 
overall  Army  objective  of  conducting  combat  operations.  It  would  be 
just  as  valid  to  view  the  tradeoffs  from  the  dual  position  of  both 
peacetime  and  wartime.  Each  of  these  perspectives  helps  define  a 
value  system.  For  example,  suppose  as  a  decisionmaker,  you  have  an 
additional  million  dollars  to  spend  on  procuring  either  Special 
Operations  Forces  (SOF)  aircraft  or  additional  Ml  tanks.  Assume  SOF 
aircraft  cost  a  third  as  much  as  the  Ml  and  is  equally  effective  in 
its  mission.  Furthermore,  as  the  decisionmaker,  if  you  think  the 
next  war  to  be  fought  will  be  an  insurgency  in  a  specific  mountainous 
theater,  then  likely  you  will  value  SOF  aircraft  over  tanks.  If, 
however,  you  believe  the  next  battle  will  be  an  armor-heavy  fight  on 
the  plains  of  Europe,  then  tanks  will  be  valued  over  SOF  aircraft. 

(3)  It  was  clear  that  the  Army  leadership  viewed  the  value  of 
program  alternatives  in  terms  of  their  contribution  to  combat.  In 
conducting  interviews  with  decisionmakers,  it  became  obvious  to  the 
study  tecun  that  decisionmakers  other  than  the  sponsor  viewed  the 
problem  from  a  sightly  different  perspective.  There  are  other 
interest  groups  such  as  Congress,  the  Executive  Branch,  or  the  Office 
of  the  Secretary  of  Defense  which  may  view  the  PPBES  process  somewhat 
differently.  These  "non-Army"  groups  may,  in  fact,  view  the  armed 
forces  as  an  agent  of  peaceful  change  within  our  own  society.  These 
groups,  although  ultimately  interested  in  the  ability  of  the  Army  to 
conduct  combat  operations,  may  be  influenced  by  other  objectives  such 
as  budget  constraints,  power  projection,  or  social  change.  This  fact 
not  only  reinforces  the  general  concept  of  context,  but  points  out 
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the  need  to  categorize  decisionmakers  by  interest  group.  Each 
interest  group  will  have  its  own  biases,  criteria,  and  issue 
relationships . 

(4)  The  idea  of  identifying  interest  groups  and  their  value 
structure  is  only  one  aspect  of  the  concept  of  context.  Another,  and 
equally  important,  aspect  is  the  need  to  define  the  parameters  of  the 
issue.  This  process  includes  identifying  the  policies,  systems, 
organizations,  and  other  factors  which  influence  the  decisionmaking 
process.  Policy  statements  will  influence  the  choice  made  through 
identifying  objectives  or  requirements  which  must  be  met.  For 
example,  analysis  of  Army  energy  investment  alternatives  are 
influenced  by  provisions  in  Title  10  of  rhe  US  Code.  Weapon  systems 
and  Army  organizations  provide  a  physical  and  structural  context. 

For  example,  the  Army  may  wish  to  look  at  the  modernization  of  the 
antitank  systems  for  the  forward-deployed  forces.  And  finally , there 
is  a  multitude  of  other  items  which  may  affect  a  decision  including 
such  things  as  the  economy,  a  leader's  world  view  (pre-CFE  versus 
parity  versus  regional  conflicts  as  the  most  likely  war  to  be  fought) 
or  his  (the  decisionmaker)  position  in  the  organization. 

(5)  Chapter  4  (Issue  Clarification)  and  Chapter  7  (Implicit 
Effectiveness)  specifically  address  this  idea  of  context.  Chapter  4 
focuses  on  the  need  to  clearly  frame  the  issue  in  a  way  that  all 
decisionmakers,  regardless  of  their  interest  group,  have  a  common 
understanding  of  the  issue  parameters.  Chapter  7  attempts  to  capture 
the  decisionmaker's  criteria  for  judging  tradeoffs  and  at  the  same 
time  trying  to  minimize  bias. 

b.  Measuring  Value 

(1)  As  the  study  team  conducted  its  literature  search  and 
reviewed  associated  work,  it  became  evident  that  any  Value  Added 
concept  must  include  a  means  to  measure  value.  Two  documents  in 
particular  influenced  the  study  team  in  developing  the  Value  Added 
Analysis  methodology:  a  US  Army  Concepts  Analysis  report  (Study  of 
Effects  of  Alternate  Allocation  of  Army  Dollar  Resources  at  Various 
Budget  Levels  -  Phase  II  -  Final  Report),  and  a  TRADOC  White  Paper 
(Combined  Hierarchical  Assessment  and  Capital  Budgeting  Framework, 

Ft.  Leavenworth,  KS.).  A  major  contributor  to  the  CAA  document  was 
particularly  helpful  in  relating  his  observations  and  in  discussing 
the  need  to  include  sufficient  capability  for  decisionmakers  to 
influence  the  decision  when  using  the  VAA  methodology.  Dr. 

Anderson's  white  paper  was  helpful  in  assisting  the  team  in  devel¬ 
oping  a  construct  for  measuring  value.  Furthermore,  the  need  to 
incorporate  a  means  by  which  subjective  decisionmaker  judgment  could 
influence  the  process  beccune  a  fundeunental  requirement  for  the 
methodology.  Often,  attempts  to  develop  a  program  development  and 
tradeoff  methodology,  such  as  Value  Added,  failed  because  these 
methodologies  concentrated  on  quantifiable  measures  of  effectiveness 
almost  exclusively.  These  preceding  methodologies  did  not  accom¬ 
modate  those  other  subjective  factors  which  decisionmakers  felt  were 
important.  This  issue  eventually  was  addressed  by  the  development  of 
two  categories  of  effectiveness  called  explicit  effectiveness 
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(directly  quantifiable)  and  implicit  effectiveness  (subjective  or 
indirectly  quantifiable). 

(2)  The  problem  of  how  to  measure  value  contextually  was  an 
essential  issue  during  the  development  of  the  Value  Added  method¬ 
ology.  Furthermore,  the  study  teaun  knew  that  the  methodology 
required  a  means  of  capturing  subjective  decisionmaker  judgment. 
Solving  these  problems  motivated  the  development  of  a  judgment 
freunework  and  two  evaluation  approaches  for  separately  measuring 
explicit  and  implicit  effectiveness. 

(a)  The  analytical  hierarchy  technique  is  used  to  develop  a 
freunework  for  weighting  the  importance  of  the  explicit  and  implicit 
measures  of  effectiveness.  These  subjective  weights  are  used  to 
depict  (or  capture)  the  decisionmaker’s  value  system.  The  two 
evaluation  approaches  are  then  used  to  measure  an  alternative's 
capability. 

(b)  The  study  team  calls  the  approach  for  measuring  explicit 
effectiveness  the  hierarchical  measures  of  effectiveness  (MOEs). 

This  approach  provides  a  means  of  quantifying  the  value  of  an  issue 
across  functional  areas  so  that  each  alternative  can  be  evaluated  in 
the  appropriate  context  by  the  appropriate  analytical  tooi.  Even 
though  this  study  uses  a  combat  example,  the  concept  applies  to 
noncombat  issues  as  well. 

(c)  The  approach  used  to  measure  the  implicit  (subjective) 
factors  uses  a  subjective  scoring  system.  Each  alternative  is  scored 
by  subject  matter  experts  against  a  set  of  criteria  developed  as  part 
of  the  analytical  hierarchy  work. 

c.  Data  Requirements 

(1)  One  fact  became  very  clear  to  the  study  team  regarding 
data  requirements  for  the  Value  Added  Analysis.  Data  for  this 
analysis  was  going  to  be  complex,  extensive,  and  varied.  This  fact 
alone  became  the  driving  factor  in  searching  for  a  flexible  computer 
architecture  and  environment  for  the  Value  Added  Analysis. 

(2)  Our  current  thought  about  the  Valued  Added  Analysis 
separates  our  data  requirements  into  two  categories — quantitative  and 
narrative.  These  two  categories  cover  a  wide  spectrum  of  data  types 
ranging  from  cost  data,  which  is  mostly  numerical,  to  policy  data, 
which  is  mostly  text.  Furthermore,  the  in:,egration  requirements 
across  the  two  categories  are  both  horizontal  and  vertical.  Keys  may 
be  as  simple  as  the  title  of  tne  alternative  or  as  complex  as  linking 
appropriation  pieces  for  the  alternative  with  other  related 
alternatives  or  issues. 

(3)  The  data  collected  for  Value  Added  Analysis  serves  two 
purposes.  First,  the  spectrum  of  data  must  be  sufficiently  rich 
across  functional  areas  to  help  inform  (educate)  the  decisionmaker 
using  VAA  about  the  issues  and  choices.  Access  to  the  alternative's 
definition  (performance  specifications,  support  requirements,  and 
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related  equipment/systems),  the  issues  associated  with  the 
alternative( s ) ,  cost  data,  organizational  implications,  and 
applicable  policy  data  must  be  simple  and  easy  to  use.  Second,  the 
data  must  be  in  sufficient  detail  to  feed  the  VAA  models. 

2-5.  SUMMARY.  The  VAA  Study  was  designed  to  develop  a 
methodology  to  conduct  program  alternative  tradeoffs.  This 
report  describes  a  comprehensive  cost-benefit  methodology  which 
enables  the  conducting  of  these  program  issue  tradeoffs.  As  a 
means  of  illustrating  the  methodology.  Chapter  3  introduces  an 
armor/ antiarmor  example  tradeoff  issue  which  is  carried  through¬ 
out  the  remaining  chapters  of  the  report.  A  case  study  of  the 
Value  Added  Analysis  methodology  using  realistic  issues  and  data 
is  presented  in  Appendix  E 
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CHAPTER  3 

VALUE  ADDED  ANALYSIS  METHODOLOGY 

3-1.  INTRODUCTION.  The  purpose  of  this  chapter  is  to  pro¬ 
vide  a  comprehensive  discussion  of  the  Value  Added  methodol¬ 
ogy.  Subsequent  chapters  (5-9)  will  present  a  more  detailed 
discussion  of  those  portions  of  the  methodology  that  were  more 
fully  implemented  during  this  study.  This  chapter  is  struc¬ 
tured  as  follows:  (1)  overview  of  the  methodology;  (2) 
description  of  each  module  in  the  methodology;  and  (3) 
summary. 

3-2.  OVERVIEW 


a.  Introduction.  The  first  objective  of  this  study,  as 
outlined  in  the  study  directive,  was  to  formulate  an  analytic  method 
for  estimating  the  marginal  value  of  competing  Army  program  alter¬ 
natives.  The  VAA  methodology  has  been  developed  to  provide  a  road 
map  for  conducting  these  analyses.  The  methodology  provides  a 
generic  approach  for  cost-benefit  analysis.  An  important  feature  of 
this  methodology  is  that  it  is  flexible  and  may  be  used  in  both  a 
standardized  or  ad  hoc  approach.  The  VAA  standardized  approach  would 
use  the  complete  methodology  as  described  in  this  chapter,  to  include 
all  of  the  current  techniques  developed.  The  ad  hoc  approach  uses 


RESOURCE 

ALLOCATION 

MODULE 


RESULTS  & 
DISPLAY 
MODULE 


RriutU 

PfWWUtOfI 


Figure  3-1.  Value  Added  Analysis  Methodology  Overview 


only  those  modules  and  techniques  pertinent  to  the  issue  being  inves 
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tigated.  This  flexibility  is  achieved  by  using  a  modular 
methodological  framework  .  Figure  3-1  depicts  the  complete 
"standardized"  VAA  methodology  with  its  eight  modules. 

b.  VAA  Modules.  Each  of  the  modules  is  designed  to  be  self- 
contained  in  that  they  can  stand  alone  if  needed.  This  autonomous 
structure  allows  the  analyst  the  flexibility  to  use  the  methodology 
in  its  entirety  (standardized  approach)  or  to  customize  the 
methodology  in  such  a  way  that  it  addresses  the  needs  of  the  current 
issue  tradeoff  analysis.  The  following  is  a  listing  of  the  modules 
contained  in  the  standardized  VAA  methodology  and  a  short  description 
of  each; 

(1)  Issue  Clarification.  This  module  is  the  initial  step 
in  the  VAA  methodology  and  consists  of  defining  and  understanding  the 
issue . 


(2)  Explicit  Effectiveness.  The  explicit  effectiveness 
module  is  divided  into  three  submodules  whose  purpose  is  to  measure 
the  direct  or  quantitative  components  of  value.  The  explicit 
contribution  of  a  program  alternative  is  defined  by  the  MOE  derived 
from  the  combat,  soldier  quality  of  life,  or  other  Army  objectives 
submodules  (see  Figure  3-1). of  the  explicit  effectiveness  module. 

(3)  Costing.  The  costing  module  assists  in  identifying  and 
calculating  the  costs  associated  with  each  alternative  for  the  issue 
being  investigated. 

(4)  Implicit  Effectiveness.  The  implicit  effectiveness 
module's  purpose  is  to  measure  the  qualitative  or  subjective 
components  of  value  through  the  use  of  leadership  surveys  and  subject 
matter  experts.  The  implicit  contribution  of  a  program  alternative 
is  defined  by  a  set  of  secondary  impact  analysis  modifiers  which  in 
effect  serve  as  implicit  MOE. 

(5)  Effectiveness  Integration.  Both  explicit  and 
implicit  measures  of  value  are  combined  within  the  effectiveness 
integration  module  to  develop  a  single  measure  of  benefit. 

(6)  Optimization.  The  marginal  value  of  program  alter¬ 
natives  is  calculated  in  the  optimization  module.  This  calculation 
is  accomplished  by  relating  the  Value  Added  coefficients  developed  in 
the  effectiveness  integration  module  with  the  costs  developed  in  the 
costing  module  according  to  established  cost-benefit  criteria.  These 
cost-benefit  criteria  specify  the  appropriate  conditions  of 
optimality. 

(7)  Resource  Allocation.  The  resource  allocation  module 
is  used  to  distribute  the  optimized  mix  of  program  alternatives  to 
the  Army  force  structure  and  organizations. 

(8)  Result  and  Display.  The  purpose  of  this  module  is  to 
present  the  result  of  the  analysis. 
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c.  VAA  Methodology  Structure.  The  dotted  lines  outlining 
various  parts  of  Figure  3-1  are  used  to  graphically  depict  the  eight 
modules  of  the  methodology.  Each  module  is  defined  by  a  process, 
technique,  input,  assumptions,  and  output.  Appendix  D  contains  a  set 
of  two  matrices  which  outlines  these  elements  for  each  module.  This 
modularity  is  an  important  aspect  of  the  VAA  methodology  because  it 
allows  the  analyst  to  investigate  wide-ranging  problems  and  issues 
which  are  inherent  in  the  PPBES  process.  Although  specific  data, 
models,  and  techniques  may  change  from  analysis  to  analysis,  the 
approach  and  the  general  form  of  each  module  remain  unchanged. 

d.  VAA  Methodology  Concept.  As  currently  envisioned,  the  VAA 
methodology  concept  characterizes  a  Value  Added  study  or  analysis  as 
either  using  a  standardized  or  an  ad  hoc  approach.  This  distinction 
is  important  because  it  determines  how  the  VAA  methodology  will  be 
used.  In  the  standardized  approach,  all  modules  and  techniques  of 
the  methodology  as  described  in  this  report  would  be  implemented. 

The  standardized  approach  would  most  likely  be  used  by  action 
officers  for  the  more  traditional  program  tradeoff  exercises,  such  as 
the  PEG  (Program  Evaluation  Group)  process,  or  POM  issue  defense.  In 
an  ad  hoc  approach,  the  analyst  would  use  only  those  modules  and 
techniques  which  are  pertinent  to  the  issue  being  investigated.  An 
example  of  this  type  study  might  be  the  FY  90  QUICKSILVER  reviews. 

e.  Standardized  VAA  Methodology.  The  current  standardized 
VAA  methodology  contains  eight  modules,  and  within  them,  there  are 
four  generic  types  of  models:  expert  systems,  simulation,  decision 
support,  and  optimization.  All  of  the  models  use  existing  and  well- 
accepted  operations  research  techniques.  The  VAA  methodology 
purposely  relies  on  analytical  tools  which  are  well  known  in  order  to 
ease  concern  over  modeling  and  computational  adequacy.  For  purposes 
of  this  report,  any  reference  to  the  VAA  methodology  will  pertain  to 
the  standardized  approach  unless  specific  reference  is  made  to  the  ad 
hoc  approach  instead.  Each  module  of  the  Value  Added  Analysis 
methodology  is  discussed  in  its  own  paragraph  of  this  chapter  except 
for  the  three  effectiveness  modules  which  are  discussed  jointly  in 
paragraph  3-4. 

3-3.  ISSUE  CLARIFICATION  MODULE 

a.  Introduction.  The  first  module  in  the  Value  Added  Analysis 
methodology  is  issue  clarification.  The  first  and  most  important 
step  for  the  VAA  methodology  is  to  spend  some  time  identifying  the 
elements  of  the  tradeoff  issue  and  the  issue-related  questions  and 
problems.  In  the  prototype  work,  the  study  team  has  seen  time  and 
again  the  importance  of  this  step.  If  this  issue  clarification  step 
is  not  thorough  and  fully  understood,  the  entire  analysis  from  this 
point  will  be  misdirected  at  best  and  irrelevant  at  worst.  Figure 
3-2  is  an  example  of  an  initial  VAA  issue. 
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Representative  Armor/Antiarmor  Systems 


Issue:  Given  limited  dollar  resources,  which  mix  of  A3  systems 
should  be  procured? 


Base  case  Europe  Scenario  6.3  with  25  systems 

•  AMS-H 

•  BRADLEY  BLK  III 

•  M1A2 

•  MLRSSADARM 

•  NLOS-AT 


Figure  3-2.  Example  VAA  Issue 

In  this  example,  the  analyst  is  asked  to  conduct  a  tradeoff  analysis 
to  select  the  best  mix  of  armor /anti armor  systems.  The  analyst  is 
provided  guidance  in  the  form  of  a  cost  constraint,  a  European 
scenario  (context),  and  a  description  of  the  baseline  force  and  two 
proposed  alternative  weapon  system  mixes.  The  alternatives  were 
developed  as  part  of  the  Armor  Antiarmor  Study  conducted  at  HQDA  by 
the  A3  task  force.  These  two  particular  alternatives  were  selected  by 
the  task  force  as  representative  mixes  and  not  necessarily  to  be 
perscriptive .  Both  alternatives  were  developed  from  a  single  list  of 
systems.  The  A3  example  will  be  used  from  this  point  on  in  the 
report  to  help  illustrate  the  VAA  methodology.  In  the  final  anal¬ 
ysis,  we  will  actually  be  trading  off  the  single  list  of  systems  tc 
provide  what  would  amount  to  a  VAA  alternative. 

b.  Processes.  The  issue  as  depicted  above  may  seem  clear,  but 
in  terms  of  the  VAA  methodology  more  is  required.  There  are  three 
processes  associated  with  the  clarification  module.  The  processes 
are: 

*  Obtain  Broad  Guidance 

*  Describe  Issue  Relationships 

*  Determine  how  the  Methodology  Will  be  Used 

(1)  Obtain  Broad  Guidance.  Figure  3-2  does  not  indicate 
what  guidance  or  factors  may  be  important  to  the  decisionmaker  in 
conducting  this  particular  Value  Added  Analysis.  The  analyst  must 
obtain  answers  to  such  questions  as:  is  there  a  budget  cut  looming? 
Do  we  need  to  consider  a  particular  investment  strategy?  What 
theaters  and  OPLANs  do  we  need  to  consider,  if  any?  These  kinds  of 
questions  must  be  formulated  and  discussed  by  the  analyst  with  the 
appropriate  set  of  decisionmakers  in  order  to  obtain  and  clarify  any 
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guidance  he  needs.  Additionally,  the  analyst  needs  to  conduct  a 
thorough  review  of  policy  statements  that  may  impact  upon  the  issue 
being  investigated. 

(2)  Describe  Issue  Relationships.  Describe  issue  rela¬ 
tionships  by  system,  organization,  policy,  and  appropriation.  At 
this  point,  the  Value  Added  methodology  requires  a  clear  description 
of  the  systems  involved  in  each  alternative,  a  breakout  of  other 
related  systems  with  their  equipment,  costs,  and  other  factors  which 
may  impact  on  the  decision.  For  example,  the  analyst  should  ask  such 
questions  aj,  "Are  there  force  development  issues  or  Congressional 
appropriation  guidelines  which  impact  upon  this  analysis?"  Table  3-1 
provides  some  examples  of  the  types  of  relationships  the  analyst 
might  explore  during  the  issue  clarification  step  of  the  illustrative 
A3  Value  Added  Analysis. 

Table  3-1.  A3  Issue  Clarification 


Issue  Relationships 

System 

Procurement  Data 

System  Definition 

Capabilities 

Orcranization 

Fielding 

Force  Structure 

Policy 

rwnTHTTITM 

Defense  Guidance 

Appropriation 

Dollar  Cost 

Manpower 

TOA  Guidance 

Each  of  the  alternatives  and  the  base  case  would  be  characterized 
using  the  four  categories  listed  in  the  column  on  the  left  of  the 
table.  Figure  3-3  depicts  (for  three  alternatives  of  the 
armor/antiarmor  issue)  a  rudimentary  example  of  the  complexity  of  the 
data  required  to  conduct  a  Value  Added  Analysis  and  the  form  in  which 
that  data  might  be  presented.  Procurement  data  is  numerical  and 
relates  to  the  production  line  capacity  and  the  ability  of  the 
industrial  base  to  produce  the  quantities  desired.  Cost  data  is 
subdivided  into  appropriation  categories  in  order  to  relate  it  in 
PPBES  terms.  The  major  item  system  map  (MISM)  tree  (alphanumeric 
data)  refers  to  a  detailed  breakdown  of  the  system  and  associated 
items  of  equipment  and  provides  a  standard  definition  of  the  system 
for  costing  and  modeling.  Other  pieces  of  information,  such  as  the 
AMM  priority  and  short  notes,  provide  policy  guidance  and  are  both 
numeric  and  text  in  character.  From  this  chart  it  should  become 
readily  apparent  that  the  information  needs  of  the  Value  Added 
Analysis  are  varied  and  complex. 


^AMM  Priority  -  Army  Modernization  Memorandum  priority  is  developed  through  the  Concepts  Based 
Requirements  System  conducted  at  TRADOC  in  support  of  the  Long  Range  Research  and  Development 
Modernization  Plan  (LRRAMP). 
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Information  Matrix:  A3  Alternatives 


System 

Attribute 

Tank  A 

Antitank 

Missile  A 

Submunition  B 

Procurement  Data 

Number  ot  Lines 

2 

1 

5 

Number  of  Shifts 

I 

2 

1 

Hours/Shift 

8 

8 

8 

Systems/Month 

180 

54 

68 

Cost 

RDTE($M) 

170 

58 

43 

OMA($M) 

14.7 

4i 

1.2 

Spaces(P:rs/TJnit) 

NA 

-58 

NA 

MISM  Tree 

UN 

B54690 

GA0167 

C67900 

E66300 

■VMM 

Priority 

i 

44 

23 

Short  Notes 

These  alternatives  address  shortfalls  in  maneuver, 
ditea  fire,  and  counterfire  for  the  close 
combat  heavy  functional  area. 

Figure  3-3 


A3  Alternative  Information  Requirements 


(3)  Determine  How  the  Methodology  Will  be  Used. 

Determine  the  degree  or  extent  to  which  this  application  of  the 
methodology  will  be  standardized  or  ad  hoc.  Not  until  the  issue  is 
fully  understood  should  the  analyst  decide  on  one  of  the  two 
generalized  forms  of  the  methodology.  This  decision  is  a  function  of 
not  only  the  issue  itself,  but  of  the  particular  analysis  objective, 
data,  models,  and  other  input  items  to  the  methodology. 

3-4.  COSTING  MODULE 

a.  Introduction.  The  second  module  in  the  Value  Added  Analysis 
methodology  pertains  to  costing.  This  module  is  intended  to  identify 
all  appropriate  costs  associated  with  each  alternative  for  the  issue 
being  investigated.  These  costs  will  usually  be  monetary  but  are  not 
limited  to  dollars  alone.  Other  measures  of  cost  may  be  included 
such  as  time  lost,  or  force  structure  reductions/increases,  and  so 
forth.  This  module  also  includes  three  sections  similar  to  those 
found  in  the  explicit  effectiveness  module;  develop  a  context, 
select  or  develop  appropriate  analytical  models  and  tools,  and 
implement  analytical  models  and  tools.  Matrix  1  in  Appendix  D 
depicts  the  full  framework  for  the  costing  module. 
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b.  Develop  A  Context.  The  first  section  of  the  costing  module 
is  used  to  develop  the  costing  context  for  the  alternatives  being 
investigated  as  part  of  this  particular  analysis.  Three  processes 
are  associated  with  the  section  on  “develop  cost  context."  The 
processes  are; 

*  Identify  Assumptions 

*  Identify  Cost  Policy 

*  Determine  Lexicon 

(1)  Identify  Assumptions.  Develop  a  list  of  assumptions 
regarding  the  costing  of  the  alternatives  to  be  investigated. 

Identify  costs  which  must  be  developed  from  surrogate  information  or 
analogous  costs.  Identify  those  alternatives  which  can  use 
cost/quantity  relationships  and  those  which  require  different 
techniques  for  costing. 

(2)  Identify  Cost  Policy.  Identify  appropriate  cost 
policies  and  their  associated  sources.  Determine  if  the  policies  are 
relevant  and  applicable  to  the  issue  being  investigated. 

(3)  Determine  Lexicon.  Identify  and  understand  the  cost 
terminology  and  coding  to  be  used  as  part  of  the  analysis.  Define 
the  resolution  and  composition  of  the  costing  to  be  conducted. 

c.  Select  or  Develop  Appropriate  Analytical  Models  and 
Tools.  The  second  section  of  the  costing  module  is  exercised  in 
order  to  determine  the  appropriate  models  and  tools  to  be  used  in 
investigating  the  alternatives  being  analyzed.  Five  processes  are 
associated  with  the  second  section.  The  processes  are; 

*  Identify  Selection  Criteria 

*  Determine  Lexicon 

*  Determine  Measures  of  Effectiveness 

*  Review  Alternative  Costing  Methods 

*  Select  Costing  Models  and  Tools 

(1)  Identify  Selection  Criteria.  Determine  attributes 
needed  in  the  costing  model(s)  required  for  this  analysis. 

(2)  Determine  Measures  of  Effectiveness.  Identify  model 
outputs  required  to  conduct  the  analysis, 

(3)  Review  Alternative  Costing  Methods.  Conduct 
literature  review  for  appropriate  existing  models  and  tools  and 
determine  feasibility  for  use  in  the  analysis. 

(4)  Select  Costing  Models  and  Tools.  Apply  selection 
criteria  in  order  to  select  appropriate  models  and  tools  for  the 
analysis . 

d.  Implement  Analytical  Models  and  Tools.  The  third 
section  of  the  costing  module  implements  the  models  and  tools 
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selected  in  section  two.  Seven  processes  are  associated  with  the 
second  section.  The  processes  are; 

*  Identify  Data  Requirements 

*  Determine  Lexicon 

*  Obtain  Modeling  Capability 

*  Develop  an  Experimental  Design  for  Costing 

*  Use  Models  and  Tools 

*  Analyze  Results 

*  Present  Results  or  Pass  Data  to  Next  Module 

(1)  Identify  Data  Requirements.  Determine  attributes 
needed  in  the  costing  model(s)  required  for  this  analysis. 

(2)  Obtain  Modeling  Capability.  The  analyst  conducting  a 
Value  Added  Analysis  would  at  this  point  obtain  the  capability  to 
exercise  the  models  and  tools  selected  to  be  used  for  the  analysis. 
This  effort  may  require  learning  the  models  and  how  to  use  them,  or 
developing  from  scratch  a  new  model  or  tool.  In  the  case  of  this 
prototype  VAA  Study,  two  costing  models  were  revised  and  incorporated 
in  the  Value  Added  methodology — the  Cost  Quantity  Model  (CQM),  and 
the  Life  Cycle  Cost  Model  (LCCM).  The  CQM  is  used  to  provide  the 
average  annual  weapon  system  cost,  considering  experience  and 
production  rate  curves. (e.g. ,  procurement  cost)  for  input  into  FOMOA, 
CQM  is  used  again  to  cost  the  optimal  annual  quantities  computed  by 
FOMOA.  The  LCCM  can  then  be  utilized  to  estimate  the  life  cycle 
costs  for  the  P92  cost  elements,  which  are  guidelines  established  by 
the  US  Army  Cost  and  Economic  Analysis  Center  (CEAC)  for  generating 
baseline  cost  estimates  (BCE) .  A  more  detailed  explanation  of  the 
models  can  be  found  in  Chapter  6. 

(3)  Develop  an  Experimental  Design  for  Costing.  The 

cost  analyst  (or  analysis  team)  needs  to  develop  an  experimental 
design  based  on  the  alternatives  to  be  investigated.  The  experi¬ 
mental  design  step  becomes  more  important  if  the  analyst  is  working 
with  costs  which  are  not  well  understood,  such  as  costs  for 
developmental  systems.  If  one  could  think  of  costs  as  a  continuum 
with  the  left  side  being  costs  which  are  well  known  (such  as 
historical  data,  through  empirical  data,  to  the  right  side  which  may 
consist  of  cost  estimates),  then  the  importance  of  a  good  experi¬ 
mental  design  becomes  evident.  The  analyst  needs  to  determine  the 
appropriate  experimental  design  using  standard  statistical 
techniques . 

(4)  Use  Models  and  Tools.  The  analyst,  using  the 
experimental  design  developed  in  the  previous  step,  conducts  the 
appropriate  model  runs  or  calculations.  The  analyst  should 
organize  the  results  in  a  logical  framework  for  displaying  and 
documenting  the  results. 

(5)  Analyze  Results.  For  the  costing  module  there  is  a 
need  to  ensure  that  all  costing  elements  are  included  and  understood. 
Because  some  of  the  costs  are  developed  through  modeling,  it  is 
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important  for  the  analyst  to  conduct  some  "common  sense"  checks 
against  a  sample  of  more  rigorously  obtained  costing  results. 

(6)  Present  Results  or  Pass  Data  to  Next  Module.  As 
indicated  in  step  5,  the  analysis  team  must  analyze  and  save  the 
results  of  the  modeling  work  for  later  presentation.  The  results 
should  be  prepared  in  two  forms.  First,  the  analyst  should  prepare 
values  for  each  MOE  by  each  alternative  to  be  passed  to  the 
optimization  module.  Secondly,  the  analyst  should  develop  a 
"traditional"  presentation  of  the  modeling  work  consisting  of  vu- 
graph  slides,  graphs,  and  charts. 

3-5.  EFFECTIVENESS 

a.  Introduction 

(1)  This  paragraph  of  Chapter  3,  unlike  the  other  paragraphs, 
combines  the  discussion  of  three  of  the  VAA  methodological  modules — 
Explicit  Effectiveness,  Implicit  Effectiveness,  and  Effectiveness 
Integration.  The  modules  are  combined  under  the  heading  of 
effectiveness  because  of  their  close  linkage  to  one  another  and  the 
manner  in  which  they  interact  within  the  overall  value  assessment 
framework.  As  the  Value  Added  Analysis  Study  progressed,  it  became 
clear  that  there  were  two  components  of  issue  effectiveness  which 
were  instrumental  in  the  decision  process.  These  components  of 
effectiveness  may  be  generally  characterized  as  being  either  explicit 
(quantifiable)  or  implicit  (qualitative  or  subjective) .  The  implicit 
measures  of  effectiveness  may  be  thought  of  as  modifiers  or 
supplemental  to  the  explicit,  and  therefore  "secondary.” 

(2)  The  Value  Added  Analysis  concept  uses  a  hierarchical 
assessment  framework  for  developing  a  single  measure  of  benefit. 

This  freunework  is  used  in  VAA  to  provide  the  logical  skeleton  in 
which  to  build  an  alternative's  effectiveness  value.  The  analytical 
hierarchy  process  (AHP)  is  particularly  effective  in  building  a 
decision  framework  and  for  eliciting  experts'  subjective  judgments. 
Figure  3-4  outlines  the  current  analytical  hierarchy  for  a 
standardized  Value  Added  Analysis. 
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ANALYTICAL 

HIERARCHY 


Value  Added 

Eftectiveness 

Coefficients 


Level  1 .0 


Combat 

Soldier  Quality 

Other  Army 

Secondary  Impact 

Effectiveness 

of  Life 

Objectives 

Analysis  Modifiers 

Level  2.0 


Level  3.0 


Figure  3-4.  Alternative  One  Information  Requirements 


Each  level  of  the  hiearchy  is  weighted  by  the  decisionmaker.  Level 
1.0  represents  the  overall  objective  of  the  hierarchy,  which  is  to 
develop  Value  Added  Effectiveness  Coefficients  (the  single  measure  of 
benefit)  for  passing  onto  the  optimization  module.  The  intermediate 
boxes  (level  2.0)  are  a  hierarchical  breakdown  of  level  1.0  and  are 
used  to  represent  the  components  of  value.  The  MOE  as  represented  by 
the  lines  beneath  each  box  (level  3.0)  in  Figure  3-4  are  the  founda¬ 
tion  of  the  hierarchy  and  are  the  fundamental  measures  of  value. 

Each  of  these  MOE  is  evaluated  either  using  quantitative  models  or 
subjective  scoring.  This  hierarchy  is  used  in  conjunction  with  the 
effectiveness  integration  module  to  build  a  single  measure  of  value. 

(3)  The  first  component  of  value  (explicit  effective¬ 
ness)  is  measured  by  that  category  of  MOEs  which  lend  themselves 
to  direct  quantification.  This  first  component  of  value  is 
represented  as  the  first  three  subdivisions  of  the  2.0  level  of 
the  Value  Added  analytical  hierarchy. 

(4)  The  second  component  of  value  (implicit  effective¬ 
ness)  is  measured  by  that  category  of  MOE  which  are  qualitative 
or  subjective  in  nature.  In  analyzing  how  program  decisions  are 
actually  made  within  the  Pentagon  and  the  Department  of  the 
Army,  it  became  very  clear  that  other  factors  also  influenced 
the  decision  process.  These  factors  became  known  to  the  study 
team  as  secondary  impact  analysis  modifiers  (SIAM  factors)  and 
are  shown  in  Figure  3-4  as  the  fourth  subdivision  of  level  2.0 
of  the  hierarchy. 

b.  Explicit  Effectiveness  Module 

(1)  Introduction.  The  purpose  of  the  third  module  in  the 
Value  Added  Analysis  methodology  is  to  develop  the  explicit 
effectiveness  values.  The  Explicit  Effectiveness  Module  contains 
three  submodules.  These  submodules  are;  Combat  Effectiveness, 
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Soldier  Quality  of  Life,  and  Other  Army  Objectives.  Although  each 
submodule  measures  a  different  functional  area  of  the  Army,  they  all 
have  two  things  in  common.  First,  the  techniques  associated  with  the 
submodule  can  measure  the  output  of  an  alternative  directly,  and 
second,  each  submodule  uses  a  common  structure.  This  structure 
includes  three  sections:  develop  a  context,  select  or  develop 
appropriate  analytical  models  and  tools,  and  implement  analytical 
models  and  tools.  Matrix  1  in  Appendix  D  depicts  the  full  structure 
for  the  combat  submodule.  Even  though  there  are  three  submodules  in 
the  methodology,  only  the  Combat  Effectiveness  Submodule  will  be 
discussed  because  of  the  limited  scope  of  this  prototype  study. 

(a)  The  Combat  Effectiveness  Dubmodule  is  as  its  name 
implies.  The  techniques  used  are  the  traditional  combat  analysis 
approaches  currently  accepted  in  the  Army. 

(b)  The  second  submodule  refers  to  those  Army  programs  and 
functions  which  address  military  personnel  and  family  member  issues. 
These  include  such  issues  as  pay,  child  care,  commissaries,  medical 
care,  etc.  These  issues  must  be  considered  separately  because  they 
cannot  currently  be  measured  directly  in  any  of  the  current  combat 
approaches.  However,  their  impact  on  readiness  is  accepted  by  all  of 
the  decisionmakers  interviewed. 

(c)  The  last  submodule  refers  to  all  the  other  objectives, 
functions  and  missions  that  the  Army  is  required  to  meet.  These 
items  would  include  such  things  as  nation  building  missions,  equal 
opportunity,  sustaining  base,  etc.  These  too  are  not  easily  related 
to  combat  but  are  certainly  important  to  the  overall  Army  mission. 

(2)  Develop  A  Context.  The  first  section  of  the  modules  is 
used  to  develop  the  explicit  effectiveness  context  for  the 
alternatives  being  investigated  as  part  of  a  Value  Added  Analysis. 

Six  processes  are  associated  with  the  section  on  context  development. 
The  processes  are: 

*  Identify  Assumptions 

*  Identify  Force  Structure 

*  Identify  Appropriate  Doctrine 

*  Identify  Resources 

*  Develop  Scenarios 

*  Determine  Lexicon 

(a)  Identify  Assumptions.  Develop  a  list  of 
assumptions  regarding  explicit  effectiveness.  The  analyst  must 
understand  the  issue(s)  and  associated  alternatives  well  enough  to 
identify  critical  elements  of  information.  Furthermore,  the  analyst 
must  recognize  which  critical  elements  of  information  may  not  have 
data  available.  This  lack  of  data  must  be  handled  either  through  the 
use  of  assumptions  (which  allows  the  use  of  surrogate  data)  or 
analogous  approaches  to  data  development.  For  example,  in  the 
illustrative  A3  problem,  a  European  scenario  was  assumed  to  be  the 
most  important  world  view  and  therefore  the  one  to  use  in  these 
tradeoffs . 
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(b)  Identify  Force  Structure.  Identify  the  force 
structure  and  levels  that  are  consistent  with  the  context  being 
developed. 

(c)  Identify  Appropriate  Doctrine.  Identify  all 

appropriate  doctrine  to  be  used  or  which  would  influence  the  combat 
modeling. 

(d)  Identify  Resources.  Identify  all  the  resources  to 

be  played. 

(e)  Develop  Appropriate  Scenarios.  Develop  scenarios 

consistent  with  the  context  being  developed. 

(f)  Determine  Lexicon.  Identify  and  understand  any 
unique  terminology  associated  with  the  issue  and  alternatives. 

(3)  Select  or  Develop  Appropriate  Analytical  Models 
and  Tools.  The  second  section  of  the  module  is  exercised  in  order 
to  determine  the  appropriate  models  and  tools  to  be  used  in 
investigating  the  alternatives  being  analyzed.  Five  processes  are 
associated  with  the  second  section.  The  processes  are; 

*  Identify  Selection  Criteria 

*  Determine  Lexicon 

Determine  Measures  of  Effectiveness 

*  Review  Alternative  Analytical  Methods 

*  Select  Models  and  Tools 

(a)  Identify  Selection  Criteria.  Determine  the  set  of 
attributes  required  to  evaluate  the  applicability  of  a  model  for  the 
Value  Added  Analysis  to  be  conducted.  A  set  of  judgment  criteria 
needs  to  be  developed  to  allow  the  analyst  to  select  the  appropriate 
models  and  tools  for  the  study.  For  the  armor/antiarmor  illustrative 
example,  the  team  determined  that  a  corps-level  combat  simulation 
capable  of  measuring  the  synergistic  effects  of  weapons  systems  was 
needed.  Furthermore,  a  model  was  needed  which  had  sufficient 
resolution  and  breadth  to  adequately  reflect  changes  in  weapon  system 
characteristics  (such  as  probability  of  kill  [PK],  rate  of  fire, 
basic  load,  etc.). 

(b)  Determine  Measures  of  Effectiveness.  Identify 
model  outputs  required  to  conduct  the  analysis.  The  explicit  MOE 
have  been,  for  the  most  part,  the  traditional  criteria  for  judging 
effectiveness.  These  MOE  might  include  such  things  as  system 
exchange  ratios,  loss  exchange  ratios,  tons  of  supply  moved,  days  of 
supply,  soldiers  per  month  retained  by  category,  child  care  days, 
etc.  The  results  of  this  module  (MOE  values  for  each  alternative) 
are  passed  to  the  effectiveness  integration  module.  Table  3-2  shows 
an  example  of  output  values  from  this  module  to  be  passed  for  three 
of  the  alternatives  from  the  A3  illustrative  example. 
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Table  3-2.  Explicit  Effectiveness  Output 
(Selected  A3  Alternatives)^ 


Systems 

MOE 

COFM 

MFCM 

BFS 

M1A2 

2.39 

134.4 

39 

NLOS-AT 

3.81 

133.1 

41 

MLRS  SADARM 

2.95 

132.6 

39 

(c)  Review  Alternative  Analytical  Methods.  Conduct 
literature  review  for  appropriate  existing  models  and  tools  and 
determine  feasibility  for  use  in  the  analysis. 

(d)  Select  Models  and  Tools.  Apply  the  selection 
criteria  in  order  to  identify  appropriate  models  and  tools  for  the 
analysis . 

(4)  Implement  Analytical  Models  and  Tools.  The  third 
section  of  the  explicit  effectiveness  module  implements  the  models 
and  tools  selected  in  section  2.  Seven  processes  are  associated  with 
the  second  section.  The  processes  are: 

*  Identify  Data  Requirements 

*  Determine  Lexicon 

*  Obtain  Modeling  Capability 

*  Develop  An  Experimental  Design 

*  Use  Models  and  Tools 

*  Analyze  Results 

*  Present  Results  or  Pass  Data  to  Next  Module. 

(a)  Identify  Data  Requirements.  Based  on  the  issue 
and  alternatives  to  be  analyzed,  review  the  models  and  tools  selected 
for  input  data  requirements.  The  analyst  should  develop  a  data 
collection  plan  and  means  by  which  to  manipulate  the  data.  For  the 
A3  illustrative  example,  the  analysis  team  would  collect  scenario, 
weapon  characteristics  data,  and  force  structure  data.  As  envisioned 
by  the  study  team,  the  final  form  of  the  Value  Added  Analysis 
Decision  Support  System  will  contain  a  centralized  data  base  engine 
to  assist  in  this  step. 

(b)  Obtain  Modeling  Capability.  The  analyst 
conducting  a  Value  Added  Analysis  would  at  this  point  obtain  the 
capability  to  exercise  the  models  and  tools  selected  to  be  used  for 


^Table  3-2  shows  example  data  output  by  system.  Fhis  may  also  be  calculated  by  package.  In  out  example  the 
base  case  and  the  two  alternatives  would  be  packages.  A  selected  set  of  systems  from  the  A3  alternatives  are 
presented  here  in  order  to  show  how  a  full  matrix  might  look.  The  following  definitions  are  provided  for  the 
acronyms  found  in  the  table; 

1 .  COFM  =  Correlation  of  Forces  and  Means  (a  measure  of  Red  strength  vs  Blue  strength) 

2.  MFCM  =  Movement  Force  Center  of  Mass(a  measure  of  performan._e  based  on  the 

distance  the  center  of  mass  of  a  force  has  moved  toward  its  objective. 

3.  BFS  =  Blue  Force  Surviving  (the  number  of  blue  units  still  combat  effective 
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the  analysis.  This  effort  may  require  learning  the  models  and  how  to 
use  them,  or  developing  from  scratch  a  new  model  or  tool.  For  the 
armor /antiarmor  example,  the  study  team  developed  a  capability  to  use 
the  Corps  Battle  Analyzer  (CORBAN)  Model.  The  effort  to  obtain  this 
capability  required  the  building  of  a  modeling  team,  training  the 
team,  obtaining  physical  space,  and  obtaining  the  use  of  appropriate 
computer  hardware. 

(c)  Develop  An  Experimental  Design.  The  analyst 
(or  analysis  team)  needs  to  develop  an  experimental  design  based 
on  the  solutions  to  be  investigated.  An  essential  requirement 
is  to  determine  if  any  groupings  of  solutions  should  be 
considered.  The  issue  clarification  module  can  be  helpful  as  a 
source  for  identifying  initial  groupings.  The  groupings  may 
grow  out  of  alternative  dependencies  such  as  physical, 
operational  or  doctrinal.  The  development  of  groupings  assist 
greatly  in  scaling  down  the  size  and  extent  of  the  experimental 
design.  After  all  groupings  have  been  completed,  the  analyst 
needs  to  determine  the  appropriate  experimental  design  using 
standard  statistical  techniques. 

(d)  Use  Models  and  Tools.  The  analyst,  using  the 
experimental  design  developed  in  the  previous  step,  conducts  the 
appropriate  model  runs  or  calculations.  The  analyst  should 
organize  the  results  in  a  logical  framework  for  displaying  and 
documenting  the  results. 

(e)  Analyze  Results.  Always  an  important  aspect  of  any 
study,  this  step  is  extremely  important  for  the  Value  Added  Analysis 
methodology.  At  first  reading,  one  may  feel  that  the  VAA  methodology 
only  requires  a  set  of  output  numbers.  The  methodology  does  indeed 
require  the  results  to  be  boiled  down  to  MOE  values  (for  passing  to 
follow-on  modules).  However,  the  reasons  for  the  MOE  values  must  be 
identified  and  kept  visible  and  available  to  provide  detailed 
explanations  of  the  results.  Detailed  results  are  always  important 
for  the  insights  they  provide,  but  in  the  case  of  the  VAA 
methodology,  the  detailed  results  specifically  become  important  as  a 
means  of  building  decisionmaker  confidence  in  the  VAA  process 
results . 


(f)  Present  Results  or  Pass  Data  to  Next  Module.  As 

indicated  in  step  five,  the  analysis  team  must  analyze  and  save  the 
results  of  the  modeling  work  for  later  presentation.  The  results 
should  be  prepared  in  two  forms.  First,  the  analyst  should  prepare 
spreadsheet  values  for  each  MOE  by  each  alternative  to  be  passed  to 
the  effectiveness  integration  module.  Secondly,  the  analyst  should 
develop  a  "traditional"  presentation  of  the  modeling  work  consisting 
of  vu-graph  slides,  graphs,  and  charts.  Using  the  illustrative 
example  from  the  A3  issue,  the  analyst  would  develop  a  cost  and 
quantity  stream  presentation  by  weapon  system  by  year. 
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c.  Implicit  Effectiveness  Module 

(1)  Introduction.  The  fourth  module  in  the  Value  Added 
Analysis  methodology  develops  the  hierarchical  weights  for  both  the 
explicit  and  implicit  components  of  value  and  develops  the  implicit 
effectiveness  values.  This  module  is  intended  to  accomplish  two 
objectives.  The  first  objective  consists  of  weighting  all  the  levels 
of  the  assessment  hierarchy  and  the  individual  criteria.  The  second 
objective  consists  of  scoring  each  alternative  using  the  SIAM 
factors.  Both  of  these  objectives  are  aimed  at  capturing  the 
subjective  aspects  of  the  decisionmaking  process.  As  discussed  in 
Chapter  2,  previous  attempts  at  program  tradeoff  methodologies  did 
not  accommodate  the  decisionmaker  and  the  process  by  which  decisions 
are  made.  The  implicit  effectiveness  module  was  specifically 
designed  to  quantify  the  subjective  elements  of  ':he  decision  process 
in  a  way  that  it  could  be  used  to  conduct  tradeoff  ai.alyses.  This 
module  also  includes  three  sections  similar  to  those  found  in  the 
explicit  effectiveness  module.  Matrix  1  in  Appendix  D  depicts  the 
full  framework  for  the  Implicit  Effectiveness  Module. 

(a)  The  implicit  (subjective)  value  component  of  the  VAA 
methodology  consists  of  a  set  of  decision  criteria,  called  SIAM 
factors.  These  SIAM  factors  are  the  implicit  criteria.  There  were 
11  SIAM  factors  established  for  the  Value  Added  Analysis  and  a 
complete  discussion  of  these  factors,  including  full  definitions  for 
each,  can  be  found  in  Chapter  7.  These  factors  are  considered  to 
measure  implicit  value  because  their  intent  is  to  capture  those  items 
which  influence  the  decisionmaking  process  but  resist  being 
quantified  directly.  These  factors  are  shown  in  Figure  3-5. 
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Secondary  Impact  Analysis  Modifiers 


Contribution  to  Industrial  Preparedness 
Political  Risk 

--  Public  Opinion 
--  Congressional  Opinion 
--  Executive  Branch  Opinion 
--  Internal  Army  Opinion 
Institutional  Stability  (Amount  of  Change) 

Program  Flexibility 

Program  Difficulty 

Asset  Versatility  and  Deployability 

Operational  and  Technological  Risk 

Criticality  of  Need  as  Related  to  Current  Capability 


Figure  3-5.  Secondary  Impact  Analysis  Modifiers 

(b)  Values  for  the  SIAM  factors  were  developed  through  the 
use  of  two  surveys. 

X’  The  first  survey  was  given  to  decisionmakers  at  the 
senior  executive  service  and  general  officer  level.  The  VAA  SIAM 
factors  were  developed  prior  to  the  implementation  of  the  survey  and 
were  modified,  added,  or  deleted  as  part  of  the  weighting  process. 
Teeuns  interviewed  and  surveyed  the  Army  leadership  residing  in 
interest  groups  such  as  the  Army  Staff  (ARSTAF),  major  Army  commands 
(MACOMs),  commanders  in  chief  (CINCs),  Office  of  the  Secretary  of 
Defense  (OSD),  etc.  The  final  result  of  this  first  survey  is  a  set  of 
secondary  factors  that  have  been  weighted  as  well  as  the  remainder  of 
the  analytical  hierarchy  (see  Figure  3-4,  page  3-10). 

Z-  The  second  survey  was  given  to  a  group  different  from 
the  decisionmakers.  This  second  group  consisted  of  subject  matter 
experts  (SME).  An  individual  is  selected  to  be  part  of  this  SME 
group  because  of  his  expertise  regarding  the  systems  (or  issues). 
These  experts  are  asked  to  score  a  system  (or  issue)  on  its  own  merit 
in  relation  to  the  SIAM  factors.  The  end  result  of  this  portion  of 
the  implicit  effectiveness  module  is  a  set  of  scores  for  each  system 
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which  are  equivalent  to  the  MOE  values  coming  out  of  the  combat 
modeling. 


(c)  These  three  elements — decisionmaker  weights,  explicit 
MOE  values,  and  SME  scores — can  be  used  either  directly  by  decision¬ 
makers  or  fed  into  a  multiobjective  linear  program  to  determine  an 
alternative's  cost  effectiveness  and,  in  turn,  acquisition  strategy. 
In  either  case,  the  explicit  MOE  and  the  implicit  MOE  are  given  a 
weight  by  the  decisionmakers  as  a  means  of  rank  ordering  the  MOE  as 
to  importance. 

(2)  Develop  a  Context.  The  first  section  of  the  module  is 
used  to  develop  the  implicit  effectiveness  context  for  the  alter¬ 
natives  being  investigated  as  part  of  this  particular  analysis.  Two 
processes  are  associated  with  the  section  on  "develop  context."  The 
processes  are: 

*  Identify  Assumptions 

*  Determine  Degree  or  Extent  of  Secondary  Factors 

(a)  Identify  Assumptions.  Develop  a  list  of  assumptions 
regarding  the  alternatives  to  be  investigated.  In  the  A3  illus¬ 
trative  example,  a  key  assumption  was  the  decreasing  federal  budget 
and  therefore  the  need  to  decrement  the  procurement  accounts . 

(b)  Determine  Degree  or  Extent.  Determine  if  the 
secondary  factors  are  to  be  applied. 

(3)  Select  or  Develop  Appropriate  Analytical  Models 
and  Tools.  The  second  section  of  the  module  is  exercised  in  order 
to  determine  the  appropriate  models  and  tools  to  be  used  in  investi¬ 
gating  the  alternatives  being  analyzed.  Five  processes  are 
associated  with  the  second  section.  The  processes  are: 

*  Identify  Selection  Criteria 

*  Determine  Lexicon 

*  Determine  Measures  of  Effectiveness 

*  Review  Alternative  Methods 

*  Select  Models  and  Tools 

(a)  Identify  Selection  Criteria.  Determine  attributes 
needed  in  the  model(s)  required  for  this  analysis.  For  our  example, 
we  needed  to  capture  the  factors  which  decisionmakers  that  the 
Department  of  the  Army  felt  were  important  in  trading  off  alternative 
programs . 


(b)  Determine  Measures  of  Effectiveness.  Identify 
model  outputs  required  to  conduct  the  analysis.  In  our  illustrative 
exaimple,  this  was  the  scoring  for  the  11  SIAM  factors  on  a  scale  of  1 
to  9 . 


(c)  Review  Alternative  Methods.  ‘  Conduct  literature 
review  for  appropriate  existing  models  and  tools  and  determine 
feasibility  for  use  in  the  analysis.  During  the  building  of  the 
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prototype  methodology,  the  literature  review  resulted  in  two 
approaches,  to  include  a  decision  tree  analysis  and  Saaty’s  pairwise 
comparison  technique. 

(d)  Select  Models  and  Tools.  Apply  selection  criteria 
in  order  to  select  appropriate  models  and  tools  for  the  analysis.  We 
selected  the  pairwise  comparison  technique  because  it  was  quick, 
easily  understood  by  the  decisionmakers,  and  easy  to  implement. 

(4)  Implement  Analytical  Models  and  Tools.  The  third 
section  of  the  module  implements  the  models  and  tools  selected  in 
section  two.  Seven  processes  str  associated  with  the  third  section. 
They  are: 

*  Identify  Data  Requirements 

*  Determine  Lexicon 

*  Obtain  Modeling  Capability 

*  Develop  An  Experimental  Design 

*  Use  Models  and  Tools 

*  Analyze  Results 

*  Present  Results  or  Pass  Data  to  Next  Module. 

(a)  Identify  Data  Requirements.  Determine  attributes 
needed  in  the  model(s)  required  for  this  analysis.  The  strawman  SIAM 
factors  were  helpful  in  focusing  the  data  requirements.  It  was 
determined  that  a  survey  of  military  judgment  based  on  a  standardized 
scale  would  be  sufficient  to  meet  the  data  needs. 

(b)  Obtain  Modeling  Capability.  The  analyst 
conducting  a  Value  Added  Analysis  would  at  this  point  obtain  the 
capability  to  exercise  the  models  and  tools  selected  to  be  used  for 
the  analysis.  This  effort  may  require  learning  the  models  and  how  to 
use  them,  or  developing  from  scratch  a  new  model  or  tool.  In  our 
example,  we  created  a  standardized  survey  briefing  and  scoring 
instrument.  We  developed  a  survey  team  and  conducted  some  practice 
surveys  to  refine  the  techniques. 

(c)  Develop  an  Experimental  Design.  The  analyst  (or 
analysis  team)  needs  to  develop  an  experimental  design  based  on  the 
alternatives  to  be  investigated.  The  experimental  design  step 
becomes  more  important  if  the  analyst  is  working  with  data  which  are 
not  well  understood.  The  analyst  needs  to  determine  the  appropriate 
experimental  design  using  standard  statistical  techniques.  The 
decisionmakers  and  PA&E  and  the  technical  advisor's  office  helped  to 
identify  the  appropriate  respondents  for  this  first  cut  prototype 
effort . 


(d)  Use  Models  and  Tools.  The  analyst,  using  the 
experimental  design  developed  in  the  previous  step,  conducts  the 
appropriate  model  runs  or  calculations.  The  analyst  should 
organize  the  results  in  a  logical  framework  for  displaying  and 
documenting  the  results.  The  survey  team  conducted  the  surveys 
over  a  3-week  period. 
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(e)  Analyze  Results.  The  methodology  also  requires  the 
results  of  the  implicit  effectiveness  module  to  be  boiled  down  to  MOE 
values  (for  passing  to  follow-on  modules).  However,  the  reasons  for 
the  MOE  values  must  be  identified  and  kept  visible  and  available  to 
provide  detailed  explanations  of  the  results.  This  is  much  more 
difficult  for  this  module  than  for  the  explicit  effectiveness  because 
the  weights  and  scores  are  developed  in  the  minds  of  both  the 
decisionmakers  and  the  subject  matter  experts.  Whenever  possible, 
the  study  team  tried  to  capture  narrative  comments  regarding  the 
weighting  and  scoring  to  provide  a  flavor  of  the  reasoning  behind  the 
final  numbers  developed.  The  study  team  created  a  series  of 
spreadsheets  from  the  results  of  the  surveys  and  conducted  both 
correlation  analysis  and  consistency  analysis  on  the  data.  Appendix 
E  contains  the  detailed  results  of  this  analysis.  Because  of  the 
classification  of  the  results.  Appendix  E  is  published  separately  in 
order  to  keep  this  report  unclassified. 

(f)  Present  Results  or  Pass  Data  to  Next  Module.  As 

indicated  in  step  5,  the  analysis  team  must  analyze  and  save  the 
results  of  the  modeling  work  for  later  presentation.  The  results 
should  be  prepared  in  two  forms.  First,  the  analyst  should  prepare 
spreadsheet  values  for  each  MOE  by  each  alternative  to  be  passed  to 
the  optimization  module.  Secondly,  the  analyst  should  develop  a 
"traditional"  presentation  of  the  modeling  work  consisting  of  vu- 
graph  slides,  graphs,  and  charts.  The  teeim  created  a  composition 
spreadsheet  and  passed  the  aata  contained  therein  to  the  TOPSIS 
(Technique  for  Ordered  Preference  by  Similarity  to  Ideal  Solution) 
Model  in  the  Effectiveness  Integration  Module. 
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d.  Effectiveness  Integration  Module 

(1)  Introduction.  The  fifth  module  in  the  Value  Added 
Analysis  methodology  combines  the  explicit  and  the  implicit 
effectiveness  values  with  the  decisionmakers'  weights.  The  current 
configuration  of  the  standardized  Value  Added  Analysis  methodology 
for  this  module  uses  a  model  called  TOPSIS.  This  technique  allows  us 
to  develop  a  single  measure  of  benefit.  This  single  measure  is 
passed  as  a  prograun  value  coefficient  into  the  linear  programing 
model.  Matrix  1  in  Appendix  D  depicts  the  full  framework  for  this 
module. 


(2)  Develop  a  Context.  The  first  section  of  the  module  is 
used  to  develop  the  effectiveness  integration  context  for  the 
alternatives  being  investigated  as  part  of  this  particular  analysis. 
Three  processes  are  associated  with  the  section  on  "develop  context." 
The  processes  are; 

*  Identify  Assumptions 

*  Identify  Policy 

*  Determine  Degree  or  Extent  of  Effectiveness  Integration 

(a)  Identify  Assumptions.  Develop  a  list  of 
assumptions  regarding  the  alternatives  to  be  investigated.  The 
single  most  important  assumption  associated  with  this  module  refers 
to  the  objective  function  of  each  SIAM  factor.  The  TOPSIS  model 
requires  the  analyst  to  declare  if  the  factor  is  to  be  maximized  or 
minimized.  This  max/min  declaration  is  applied  to  all  alternatives 
and  therefore  assumes  that  the  objective  function  will  not  change 
from  one  alternative  to  the  next. 

(b)  Identify  Policy.  Identify  appropriate  policies  and 
their  associated  sources.  Determine  if  the  policies  are  relevant  and 
applicable  to  the  issue  being  investigated.  In  the  illustrative 
example,  the  objective  function  for  progreim  flexibility  was  set  to 
"maximize"  based  on  the  policy  that  programs  should  be  flexible. 

(c)  Determine  Degree  or  Extent  of  Effectiveness 
Integration.  Determine  if  the  effectiveness  is  to  be  completely 
integrated  or  partially.  If  effectiveness  to  be  incorporated  in  the 
follow-on  modules  or  extracted  at  this  point  and  provided  as  raw 
output . 


(d)  Determine  Lexicon.  Identify  and  understand  the 
terminology  to  be  used.  Ensure  that  the  max/min  relationship  for 
each  factor  is  understood  and  fully  defined. 

(3)  Select  or  Develop  Appropriate  Analytical  Models 
and  Tools.  The  second  section  of  the  module  is  exercised  in  order 
to  determine  the  appropriate  models  and  tools  to  be  used  in 
investigating  the  alternatives  being  analyzed.  Five  processes  are 
associated  with  the  second  section.  The  processes  are: 
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*  Identify  Selection  Criteria 

*  Determine  Lexicon 

*  Determine  Measures  of  Effectiveness 

*  Review  Alternative  Methods 

*  Select  Models  and  Tools 

(a)  Identify  Selection  Criteria.  Deterr»ine  attT-ibutes 
needed  in  the  effectiveness  integration  model(s)  required  for  this 
analysis.  The  need  to  combine  multiple  attributes  was  the  key 
criterion  in  selecting  a  model  for  this  module.  Furthermore,  vaa 
needed  a  way  in  which  this  combination  could  be  quantified  and 
consolidated  into  a  single  value. 

(b)  Determine  Measures  of  Effectiveness.  Identify 
model  outputs  required  to  conduct  the  analysis.  The  MOE  for  this 
module  was  set  by  EEA  2  in  which  we  hypothesized  that  a  linear 
combination  of  value  components  could  be  used  to  arrive  at  a  single 
numeric  measure  of  value.  For  this  prototype  effort,  the  TOPSIS 
value  measuring  the  distance  of  an  alternative  from  an  ideal  solution 
in  a  Euclidean  space  bec2une  this  single  measure  of  value. 

(c)  Review  Alternative  Methods.  Conduct  literature 
review  for  appropriate  existing  models  and  tools  and  determine 
feasibility  for  use  in  the  analysis.  Our  literature  review  did  not 
identify  candidates  other  than  TOPSIS. 

(d)  Select  Models  and  Tools.  Apply  selection  criteria 
in  order  to  select  appropriate  models  and  tools  for  the  analysis. 
TOPSIS  was  selected  because  it  was  available  and  met  the  need  to 
create  a  linear  combination  of  value  components. 

(4)  Implement  Analytical  Models  and  Tools.  The  third 
section  of  the  module  implements  the  models  and  tools  selected  in 
section  2.  Seven  processes  are  associated  with  the  second  section. 
The  processes  are; 

*  Identify  Data  Requirements 

*  Determine  Lexicon 

*  Obtain  Modeling  Capability 

*  Develop  an  Experimental  Design 

*  Use  Models  and  Tools 

*  Analyze  Results 

*  Present  Results  or  Pass  Data  to  Next  Module. 

(a)  Identify  Data  Requirements.  Determine  attributes 
needed  in  the  model(s)  required  for  this  analysis.  The  data  needed 
for  this  module  is  directly  provided  by  the  explicit  and  implicit 
effectiveness  modules  and  includes  the  decisionmaker's  weights,  the 
combat  simulation  results  (MOE),  and  the  scoring  of  the  11  SIAM 
factors . 


(b)  Obtain  Modeling  Capability.  The  analyst 
conducting  a  Value  Added  Analysis  would  at  this  point  obtain  the 
capability  to  exercise  the  models  and  tools  selected  to  be  used  for 
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the  analysis.  This  effort  may  require  learning  the  models  and  how  to 
use  them,  or  developing  from  scratch  a  new  model  or  tool.  The  TOPSIS 
model  was  written  in  "C“  programming  language  by  the  team  for 
inclusion  in  the  METAPHOR™^  environment  and  subsequent  VAA  Decision 
Support  System.  For  more  information  on  TOPSIS,  see  Appendix  G. 

(c)  Develop  an  Experimental  Design.  The  analyst  (or 
analysis  team)  needs  to  develop  an  experimental  design  based  on  the 
alternatives  to  be  investigated.  For  this  prototype  analysis,  we 
developed  a  design  which  investigated  the  differences  in  the  single 
measure  of  value  when  we  chose  different  MOE  for  the  explicit 
effectiveness  module.  In  particular,  there  was  concern  expressed 
regarding  the  nontraditional  MOEs  of  Blue  force  surviving,  correla¬ 
tion  of  forces  and  means,  and  movement  force  center  of  mass.  The 
te2un  looked  at  loss  exchange  ratios  (LER),  system  exchange  ratios 
(SER),  and  fractional  exchange  ratios  (FER)  as  substitutes  for  the 
MOE  we  originally  selected. 

(d)  Use  Models  and  Tools.  The  analyst,  using  the 
experimental  design  developed  in  the  previous  step,  conducts  the 
appropriate  model  runs  or  calculations.  For  our  illustrative 
excimple,  this  step  was  very  simple  and  took  a  matter  of  seconds. 

(e)  Analyze  Results.  Some  parametric  analysis  was 
conducted  on  the  input  data  (especially  the  combat  MOE)  to  gain 
insight  into  which  factors  truly  influenced  the  outcome.  For 
example,  we  used  both  the  original  set  of  explicit  MOE  (BFS,  COFM, 
and  MFCM)  and  the  fractional  exchange  ratio  to  see  how  the  range  of 
the  TOPSIS  values  would  change  for  each  alternative. 

(f)  Present  Results  or  Pass  Data  to  Next  Module.  As 

indicated  in  step  5,  the  analysis  teaim  must  analyze  and  save  the 
results  of  the  modeling  work  for  later  presentation.  The  results  for 
this  module  for  our  illustrative  example  were  prepared  in  two  forms. 
First,  a  spreadsheet  of  program  value  coefficients  for  each 
alternative  was  prepared  and  passed  to  the  optimization  module. 
Secondly,  a  "traditional"  presentation  of  the  modeling  work  was 
developed  consisting  of  vugraph  slides. 

3-6.  OPTIMIZATION  MODULE 

a.  Introduction.  The  sixth  module  in  the  Value  Added  Analysis 
methodology  uses  the  progreim  value  coefficients  developed  in  the 
Effectiveness  Integration  Module  and  the  costs  from  the  Costing 
Module  to  conduct  a  cost-benefit  analysis.  The  optimization  tool 
provides  the  means  for  integrating  the  costs  and  benefits  into  a 
comprehensive  cost-benefit  analysis.  There  is  a  shortcoming  in  the 
present  configuration  of  the  cost-benefit  analysis  We  consider  only 
procurement  costs  here.  O&S  costs  are  used  in  the  prototype  for 
accounting  purposes  only.  Furthermore  the  cost  values  used  for  the 

^  ^  METAPHOR  “  is  a  trademark  of  the  Metaphor  Computer  Corporation.  Metaphor  software  provides  an 
icon-based  user  interface  to  Standard  Query  Language  (SOL)-based  relational  data  bases. 
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optimization  constitute  average  unit  costs.  Costs  are  actually  a 
function  of  the  quantity  produced  (see  chapter  6).  No  provision  was 
made  in  this  formulation  to  take  into  account  this  functional 
relationship.  Therefore  the  possibility  exists  that  using  average 
unit  costs  as  an  approximation  could  result  in  a  violation  of  the 
budget  constraint  when  the  true  cost  is  considered.  Another  possible 
result  is  suboptimality  in  the  solution.  Unfortunately,  this  problem 
has  plagued  analysts  for  years,  and  its  solution  was  beyond  the  scope 
of  this  study.  This  problem  should  be  addressed  in  future  Value 
Added  Analysis  applications 

b.  The  cost-benefit  analysis  is  characterized  as  an  optimization 
of  an  alternative  mix  and  affordability  problem.  It  is  within  this 
module  that  the  marginal  value  of  an  alternative  is  translated  into 
meaningful  tradeoffs.  The  current  configuration  of  the  standardized 
Value  Added  Analysis  methodology  for  this  module  uses  an  optimization 
model  called  FOMOA  (Force  Modernization  Analyzer).  Matrix  1  in 
Appendix  D  depicts  the  full  framework  for  this  module. 

b.  Develop  a  Context.  The  first  section  of  the  module  is  used 
to  develop  the  optimization  context  for  the  alternatives  being 
investigated  as  part  of  this  particular  analysis.  Three  processes 
are  associated  with  the  section  on  "develop  context".  The  processes 
are: 

*  Identify  Assumptions 

*  Identify  Policy 

*  Determine  Lexicon 

(1)  Identify  Assumptions.  Develop  a  list  of  assumptions 
regarding  the  alternatives  to  be  investigated.  For  this  prototype 
study,  an  objective  of  maximizing  the  total  value  subject  to  force 
structure  and  dollar  constraints. 

(2)  Identify  Policy.  Identify  appropriate  policies  and 
their  associated  sources.  Determine  if  the  policies  are  relevant  and 
applicable  to  the  issue  being  investigated.  As  part  of  the  issue 
clarification  work  conducted  for  the  VAA  proof  of  concept  case  study 
policies  regarding  production  line  start  dates  and  capacities  were 
identified  as  key  policy  factor. 

(3)  Determine  Lexicon.  Identify  and  understand  the 
terminology  to  be  used.  Define  the  resolution  and  composition  of  the 
constraints . 

c.  Select  or  Develop  Appropriate  Analytical  Models  and 
Tools.  The  second  section  of  the  module  is  exercised  in  order  to 
aetermine  the  appropriate  models  and  tools  to  be  used  in 
investigating  the  alternatives  being  analyzed.  Five  processes  are 
associated  with  the  second  section.  The  processes  are: 

*  Identify  Selection  Criteria 

*  Determine  Lexicon 

*  Determine  Measures  of  Effectiveness 
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*  Review  Alternative  Methods 

*  Select  Models  and  Tools 

(1)  Identify  Selection  Criteria.  Determine  attributes 
needed  in  the  model(s)  required  for  this  analysis.  The  key  modeling 
attribute  needed  for  this  module  was  the  ability  to  optimize 
alternatives  using  a  cost-benefit  frcunework. 

(2)  Determine  Measures  of  Effectiveness.  Identify  model 
outputs  required  to  conduct  the  analysis.  The  MOE  selected  for  this 
module  included  ease  of  optimization  formulation,  the  ability  to 
handle  production  line  constraint  information,  a  generalized 
algebraic  formulation  which  created  a  very  flexible  and  robust  model, 
and  the  ability  to  run  on  multiple  platforms. 

(3)  Review  Alternative  Methods.  Conduct  literature 
review  for  appropriate  existing  models  and  tools  and  determine 
feasibility  for  use  in  the  analysis.  The  literature  review  for  this 
module  was  not  extensive  because  of  previous  work  completed  at  CAA. 
The  Value  Added  team  incorporated  this  previous  work  because  of  its 
direct  applicability  to  the  VAA  Study. 

(4)  Select  Models  and  Tools.  Apply  selection  criteria 
in  order  to  select  appropriate  models  and  tools  for  the  analysis.  The 
selection  criteria  were  in  part  created  as  we  reviewed  the  previous 
optimization  work  completed  at  CAA.  This  step  was,  in  effect, 
simultaneous  with  step  2  above. 

d.  Implement  Analytical  Models  and  Tools.  The  third 
section  of  the  module  implements  the  models  and  tools  selected  in 
section  two.  There  are  six  processes  associated  with  the  second 
section.  The  processes  are: 

*  Identify  Data  Requirements 

*  Obtain  Modeling  Capability 

*  Develop  An  Experimental  Design 

*  Use  Models  and  Tools 

*  Analyze  Results 

*  Present  Results  or  Pass  Data  to  Next  Module 

(1)  Identify  Data  Requirements.  Determine  attributes 
needed  in  the  model(s)  required  for  this  analysis.  The  data 
required  that  was  unique  to  this  module  was  focused  on  dollar  and 
production  line  constraint  information. 

(2)  Obtain  Modeling  Capability.  The  analyst  conducting  a 
Value  Added  Analysis  would  at  this  point  obtain  the  capability  to 
exercise  the  models  and  tools  selected  to  be  used  for  the  analysis. 
For  the  prototype  study,  the  team  used  an  optimization  approach  and 
formulation  develop  by  CAA  for  use  in  building  modernization  plans. 

We  modified  this  model  and  installed  it  on  a  Macintosh  Ilex.  The 
team  was  up  and  using  the  new  model  approximately  2  days  after 
installation . 
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(3)  Develop  an  Experimental  Design.  The  analyst  (or 
analysis  team)  needs  to  develop  an  experimental  design  based  on  the 
alternatives  to  be  investigated.  The  experimental  design  for  our 
illustrative  example  was  driven  by  the  budget  decrement  exercises 
being  conducted  at  the  Pentagon  during  the  spring  and  summer  of  1990. 
We  also  included  in  the  design  a  set  of  runs  which  would  allow  us  to 
investigate  the  sensitivity  of  the  model  to  selected  input 
parameters . 

(4)  Use  Models  and  Tools.  The  analyst,  using  the 
experimental  design  developed  in  the  previous  step,  conducts  the 
appropriate  model  runs  or  calculations.  We  conducted  two  sets 
of  model  runs  based  on  two  views  of  the  92-97  POM.  The  first 
was  based  on  a  pre-CFE  view,  and  the  second  was  based  on  a  post- 
QUICKSILVER  (post-CFE  view)  dollar  decrement. 

(5)  Analyze  Results.  For  our  illustrative  example,  we 
looked  at  the  interactions  taking  place  because  of  the  various 
constraints  and  input  data.  The  team  tried  to  isolate  the  most 
important  data  for  future  investigation. 

(6)  Present  Results  or  Pass  Data  to  Next  Module.  As 
indicated  in  step  five,  the  analysis  team  must  analyze  and  save  the 
results  of  the  modeling  work  for  later  presentation.  The  results  of 
our  illustrative  example  were  presented  as  "typical  '  funding  and 
quantity  streams  one  would  find  presented  in  the  Army  POM. 

3-7.  RESOURCE  ALLOCATION  MODULE.  The  seventh  module  in  the 
Value  Added  Analysis  methodology  uses  results  of  the  optimization 
module  to  distribute  the  mix  of  alternatives  to  the  appropriate  Army 
organization  or  agency.  For  this  prototype  study,  we  did  not  develop 
this  module.  Matrix  1  in  Appendix  D  depicts  the  full  framework  for 
this  module. 

3-8.  RESULTS  AND  DISPLAY  MODULE.  The  eighth  module  in  the 
Value  Added  Analysis  methodology  contains  the  set  of  models  and 
tools  required  to  analyze  and  present  the  results  in  a  meaning¬ 
ful  manner.  The  VAA  methodology  requires  a  flexible  data  base 
manipulator  to  provide  responsive  data  handling  for  both  output 
and  input  data.  Furthermore,  this  methodology  requires  a 
computer  environment  which  will  assist  the  analyst  in  the  task 
of  integrating  all  of  the  modules  into  a  coherent  decision 
support  system.  The  METAPHOR™  environment  has  permitted  this 
integration  by  providing  a  single  architecture  for  manipulating 
issue  description  data,  system  definition  data,  the  TOPSIS 
Model,  the  FOMOA  Model,  and  a  variety  of  display  screens,  and  by 
combining  these  elements  into  a  single  decision  support  system. 

We  have  not  developed  a  set  of  presentation  formats  nor  reports. 

Our  objective  for  this  module  during  this  prototype  study  was  to 
identify  the  requirement  and  propose  a  capability  to  be  further 
developed  in  the  future. 
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CHAPTER  4 

ISSUE  CLARIFICATION 


4-1.  INTRODUCTION 

a.  The  purpose  of  this  chapter  is  to  provide  a  discussion  of  the 
Issue  Clarification  Module  of  the  Value  Added  Analysis  methodology. 
The  chapter  addresses  the  four  processes  contained  within  this 
module.  The  chapter  is  structured  as  follows: 

(1)  Introduction 

(2)  Receive  the  Issue 

(3)  Obtain  Broad  Guidance 

(4)  Describe  Issue  Relationships 

(5)  Determine  How  the  Methodology  Will  be  Used 

b.  Figure  4-1  depicts  the  elements  of  the  module  to  include  the 
receiving  of  an  issue,  a  description  of  the  issue  in  the  context  of 
four  subeleme'’ts  (system,  organization,  appropriation,  and  policy), 
and  the  conc^  jt  of  understanding  issues. in  light  of  national  military 
goals  and  o’  jectives. 


c.  Issue  clarification  is  perhaps  the  most  important  module  in 
the  Value  Added  Analysis  methodology  because  its  purpose  is  to  define 
the  problem  that  needs  to  be  analyzed,  and,  as  such,  establishes  the 
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framework  for  the  entire  analysis.  Thus,  an  inadequate  issue 
clarification  phase  translates  directly  into  a  poorly  defined  problem 
and  a  misleading  or  irrelevant  solution. 

d.  The  actual  methodology  development  of  the  Issue  Clarification 
Module  accomplished  in  this  study  is  related  primarily  to  defining 
the  information  needed,  establishing  critical  relationships,  and 
identifying  the  types  of  analytical  tools  and  methods  that  might  be 
useful  to  perform  the  tasks  associated  with  the  module.  No  specific 
tools  or  methods  were  developed  or  used  in  this  study  for  issue 
clarification.  As  the  Value  Added  Analysis  methodology  matures, 
these  too2s  and  methods  will  be  produced  to  expedite  and  automate  the 
issue  clarification  portion  of  the  overall  methodology. 

4-2.  RECEIVE  THE  ISSUE.  An  example  of  an  issue  that  might 
be  analyzed  using  the  Value  Added  methodology  is  shown  in  Figure 
4-2. 


Example:  Representative  Armor/Anti-Armor  Systems 


Issua:  Given  limited  dollar  resources,  which  mix  of  A3  Systems 
should  be  procured? 


Alternatives  Chafacttristiw 


Basecase  Europe  Scenario  6.3  with  25  Systems 

Alt  1  •  M1A2 

•  NLOS-AT 

•  MLRS  SADARM 

Alt  2  •  M1A2 

•  AMS-H 

•  BRADLEY  BUCm 

’  MLRS  SADARM 


Figure  4-2.  Example  VAA  Issue 

Issues  received  by  the  analyst  are  likely  to  be,  at  first  cut, 
very  fuzzy.  Thus,  the  issue  clarification  step  is  inherently 
iterative.  As  the  analyst  gathers  more  and  more  information  in 
the  conduct  of  the  analysis,  systems  may  change  or  be  added  or 
deleted,  scenarios  could  be  modified,  additional  alternatives 
could  be  considered,  and  the  like.  Consequently,  the  analyst 
must  begin  with  a  broader  perspective  and,  as  more  information 
is  obtained,  narrow  the  focus  of  the  issue  to  get  to  the  heart 
of  the  matter.  This  narrowing  and  clarification  is  accomplished 
using  the  three  remaining  processes  associated  with  the  Issue 
Clarification  Module.  These  processes  are:  obtain  broad 
guidance,  describe  issue  relationships,  and  determine  how  the 
methodology  v'ill  be  used.  These  processes  will  be  described  in 
detail  in  subsequent  sections. 

4-3.  OBTAIN  BROAD  GUIDANCE.  In  order  to  provide  the  best 
possible  information  and  assistance  to  the  decisionmaker,  the 
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analyst  must  have  a  clear  idea  of  the  viewpoint  of  the  decision¬ 
maker,  his  "World  View,"  and  the  decisionmaking  environment  with 
respect  to  the  factors  that  are  most  important  to  consider.  For 
excunple,  the  decisionmaker's  opinion  as  to  the  most  probable 
next  conflict  scenario  must  be  considered  when  evaluating 
alternatives .  Another  example  involves  the  investment  strategy 
of  the  decisionmaker.  If  the  strategy  entails  more  emphasis  on 
research  and  development  and  less  on  procurement,  the  thrust  of 
the  analysis  would  be  different.  One  must  also  be  cognizant  of 
the  constraints  under  which  the  decision  will  be  made.  For 
instance,  if  Congress  has  mandated  that  a  particular  system 
should  or  should  not  be  fielded,  or  if  a  particular  program  is 
highly  unpopular  politically,  the  analysis  should  take  these 
factors  into  consideration. 

4-4.  DESCRIBE  ISSUE  RELATIONSHIPS 

a.  Introduction.  In  order  to  properly  place  an  issue  or 
group  of  issues  into  the  appropriate  context,  the  analyst  must 
be  able  to  describe  the  relationships  of  the  issue  by  system, 
organization,  policy,  and  appropriation.  Only  then  can  the 
impacts  of  the  decision  be  evaluated  in  the  broadest  sense.  In 
order  to  accomplish  this  portion  of  the  analysis,  a  significant 
amount  of  research  is  regidred  on  the  part  of  the  analyst  to 
ensure  that  all  areas  af Meeting  the  issue  and  affected  by  the 
issue  are  examined.  For  the  most  part,  this  research  must  be 
done  by  consulting  subject  matter  experts,  documents,  and  data 
bases  to  obtain  this  necessary  information.  The  possibility 
exists  that  some  expert  system  approach  to  data  identification 
and  collection  would  be  useful  in  this  process.  An  example  of 
the  type  of  software/model  combination  needed  in  this  module  is 
the  Stakeholder  software/model  developed  at  the  University  of 
Arizona,  Tucson,  as  part  of  their  Group  Decision  Support 
Laboratory.  The  models  and  software  designed  for  this  section 
of  the  module  must  help  identify  interest  groups,  categorize 
data,  and  discern  relationships  between  various  factors. 

b.  Organizational  Relationships.  Of  equal  importance 
to  the  impacts  of  a  decision  from  a  systems  perspective  are 
impacts  relating  to  organizations.  For  example,  the  theaters 
that  would  be  affected  and  the  types  of  impacts  on  those 
theaters  must  be  considered.  One  impact  might  be  the  need  to 
modify  operation  plans  (OPLANs)  as  the  result  of  the  implemen¬ 
tation  of  some  alternative.  Consideration  must  also  be  given  to 
the  units  that  are  affected  by  these  decisions. 

c.  Policy  Relationships.  Policy  relationships  are 
directly  related  to  tne  decisionmaking  environment  that  exists. 
Policy  statements  will  influence  the  choices  made  through 
identifying  objectives  or  requirements  that  must  be  met.  For 
the  Value  Added  Analysis  methodology,  this  portion  of  the  module 
should  be  closely  aligned  to  major  policy  documents  such  as  the 
Defense  Guidance  (DG),  The  Army  Plan  (TAP),  or  the  Army  Long- 
Range  Plan.  Additionally,  any  policy  decisions  which  have  been 
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specifically  made  regarding  the  issue  to  be  analyzed  must  be 
fully  understood.  One  example  of  this  type  of  policy  is  the 
decision  to  try  and  develop  armor /anti armor  systems  using  a 
common  vehicle  chassis. 

d.  System  Relationships.  The  Value  Added  Analysis 
methodology  requires  a  clear  description  of  the  systems  involved 
in  each  alternative.  The  Army  Resource  Integration  and 
Management  (ARIM>  Study  performed  by  CAA  suggests  a  method  of 
management  of  systems (AMIS  -  Army  Major  Item  Systems)  through 
the  use  of  a  comprehensive  coding  logic.  In  this  method,  each 
system  is  defined  with  respect  to  its  primary  equipment, 
associated  items  of  equipment  (ASIDE),  personnel  requirements, 
associated  facilities,  secondary  items,  and  training  items. 
Through  the  coding  schema,  the  system  definition  can  be  related 
to  its  costs,  budgeting,  force  structure  considerations,  and  the 
like.  The  ARIM  methodology  can  be  used  to  identify  the  impact, 
from  a  system  perspective,  of  the  various  issue  alternatives. 
ARIM  was  designed  for  incorporation  into  the  Value  Added 
Analysis  Methodology.  Figure  4-4  depicts  this  incorporation. 


Figure  4-3.  Value  Added/ARIM  Relationship 
4-5.  DETERMINE  HOW  THE  METHODOLOGY  WILL  BE  USED 

a.  Introduction.  The  Value  Added  Analysis  methodology  is 
comprised  of  many  components  and  modules.  Each  part  serves  a 
particular  purpose  and  provides  a  certain  portion  of  the 
information  needed  to  support  the  decisionmaking  process. 
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Factors  such  as  the  scope  of  the  study,  the  alternatives  being 
considered,  the  type  of  decision  that  needs  to  be  made,  and  the 
timeframe  in  which  study  results  must  be  provided  will  drive  the 
information  and  analysis  requirements  of  the  investigation. 

Often  these  and  other  factors  will  dictate  whether  or  not  the 
entire  methodology  will  be  utilized.  In  some  cases,  only  a  few 
of  the  VAA  modules  will  be  needed  to  respond  to  analytical 
requirements.  For  these  cases,  the  methodology  is  sufficiently 
flexible  to  allow  individual  models  and  methods  to  be  used 
separately  to  provide  input  to  the  decisionmakers.  Clearly,  a 
complete  understanding  of  the  problem  to  be  solved  is  critical 
in  determining  what  approach  should  be  used  to  attack  the 
problem.  This  understanding  will  have  been  generated  in  the 
three  processes  of  the  Issue  Clarification  Module.  This 
knowledge  can  then  be  used  to  determine  whether  a  full-scale, 
standardized,  implementation  of  the  VAA  methodology  should  be 
used,  or  if  some  reduced  scale,  ad  hoc,  method  is  appropriate. 

b.  Standardized  Methodology.  The  full-scale  imple¬ 
mentation  of  the  Value  Added  Analysis  methodology  involves  the 
coordinated  and  comprehensive  use  of  all  the  modules.  As  such, 
large  amounts  of  data  will  need  to  be  collected,  potentially 
great  numbers  of  simulation  runs  will  need  to  made,  new  surveys 
will  need  to  be  developed  and  administered,  and  all  of  the 
assembled  information  will  need  to  be  analyzed  and  checked  for 
consistency  in  order  to  use  the  rest  of  the  VAA  models  and 
tools,  and  ultimately,  to  ensure  high-quality  input  to  the 
decisionmaker.  Obviously  the  expected  duration  of  such  a  study 
effort  would  be  on  the  order  of  months,  and  a  great  deal  of 
prior  planning  is  necessary  to  produce  results  in  a  timely 
fashion.  The  standardized  implementation  of  the  VAA  methodology 
will  be  required  whenever  major  changes  in  study  assumptions, 
input  data,  or  decisionmakers  occur.  The  standardized  VAA 
methodology  is  the  subject  of  subsequent  chapters  of  this  study 
report . 


c.  Ad  Hoc  Implementation.  An  ad  hoc  implementation  of 
the  Value  Added  Analysis  methodology  is  characterized  by  less 
than  full  utilization  of  the  entire  VAA-  system.  In  these 
instances,  the  output  of  some  intermediate  module  is  sufficient 
to  answer  the  question  that  is  asked.  For  example,  a  recent 
study  was  performed  concerning  command  and  communication  systems 
where  the  issue  was  already  clarified,  and  only  the  output  of 
the  implicit  and  explicit  modules  and  the  effectiveness  inte¬ 
gration  module  was  needed.  Care  must  be  exercised,  however,  to 
ensure  that  all  questions  are  satisfactorily  addressed. 
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CHAPTER  5 

EXPLICIT  MEASURES  OF  VALUE 


5-1.  INTRODUCTION 

a.  The  purpose  of  this  chapter  is  to  provide  a  discussion  of  the 
Explicit  Effectiveness  Module  of  the  Value  Added  methodology.  The 
chapter  addresses  one  of  the  three  submodules  in  detail , Combat .  The 
other  submodules,  Soldier  Quality  of  Life  and  Other  Army  Objectives, 
are  not  discussed  in  much  detail  because  of  the  limited  scope  of  this 
first  Value  Added  Study.  These  submodules  are  included  in  the 
methodology  in  order  to  lay  out  completely  the  methodology  as 
currently  envisioned.  This  chapter  is  structured  as  follows: 

(1)  Introduction 

(2)  Combat  Effectiveness 

(3)  Noncombat  Effectiveness 

(4)  Summary 

b.  Figure  5-1  depicts  the  three  submodules  of  the  Explicit 
Effectiveness  Module.  Although  the  chart  implies  that  all  three 
submodules  are  used,  this  is  not  necessarily  true  for  all  issues 
to  be  investigated.  The  effectiveness  of  some  program  issues 
and  their  associated  alternatives  may,  in  fact,  develop  explicit 
effectiveness  values  from  all  three  of  the  submodules.  However, 
experience  has  shown  that  most  issues  will  require  only  one  or 
two  of  the  submodules  be  used  to  develop  the  explicit 
effectiveness  for  a  program  issue.  The  decision  to  use  single 
or  multiple  measures  of  explicit  effectiveness  will  grow  out  of 
the  work  completed  in  the  Issue  Clarification  Module. 

c.  Most  decisionmakers  interviewed  for  this  study  indicated  the 
need  to  explicitly  relate  issues  to  the  Army's  number  one  objective 
of  conducting  combat  operations.  However,  not  all  Planning, 
Programming,  Budgeting,  and  Execution  System  (PPBES)  issues  are 
measured  by  their  direct  contribution  to  combat  effectiveness.  Some 
issues,  such  as  base  operations  or  soldier  retention,  do  not  lend 
themselves  to  combat  simulation  or  combat-related  analysis.  Some 
PPBES  issues,  by  their  very  nature,  do  not  directly  contribute  to 
combat,  but  without  these  functions,  the  Army  would  not  be  prepared 
to  conduct  combat  operations.  It  was  from  this  fact  that  the  need  to 
measure  explicit  effectiveness  as  more  than  combat  MOE  was 
identified. 
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d.  Because  VAA  is  a  marginal  analysis,  the  MOE  are  computed  as 
incremental  changes  from  some  known  baseline.  In  the  example,  the 
baseline  is  determined  by  developing  a  base  case  scenario  for  the 
combat  simulation.  This  base  case  represents  a  known  point  such  as 
the  current  program  force  and  Red  threat  for  a  particular  fiscal  year 
(FY).  After  the  base  case  is  verified  and  accepted  \s  representative 
of  current  capability,  each  alternative  is  investigated  by  incor¬ 
porating  the  alternative  into  the  base  case.  This  new  simulation  is 
called  an  excursion.  Changes  in  the  MOE  values  as  a  result  of  an 
excursion  are  recorded.  It  is  this  incremental  change  from  the  base 
that  becomes  the  explicit  value  of  the  alternative  portrayed  in  the 
excursion. 

e.  Although  our  example  uses  only  a  combat  framework  for 
determining  the  explicit  effectiveness,  the  preceding  approach 
would  be  used  for  the  other  modules  as  well.  An  issue  being 
investigated  may  need  to  be  measured  by  one  of  the  explicit 
effectiveness  submodules,  or  two,  or  all  three.  The  decision  to 
use  one  or  multiple  submodules  will  be  based  on  identifying  all 
of  the  appropriate  MOE.  The  explicit  MOE  chosen  must  be  perti¬ 
nent  to  the  issue,  must  directly  measure  output,  and  must  be 
sensitive  enough  to  measure  changes  as  a  result  of  trading  off 
alternatives . 

5-2.  COMBAT  EFFECTIVENESS 

a.  Development  of  Combat  Context 

(1)  The  purpose  of  the  combat  effectiveness  aspect  of 
Value  Added  Analysis  is  to  develop  a  cohesive,  rational  frame¬ 
work  for  measuring  combat  effectiveness.  As  a  result  of  the 
survey  of  Army  leadership,  it  was  found  that  the  importance  of 
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combat  effectiveness  was  as  much  as  6  to  1  over  the  SIAM  factors 
and  the  noncombat  effectiveness.  Because  of  the  importance 
given  to  combat  effectiveness  by  the  decisionmakers,  it  is 
necessary  to  carefully  establish  the  environment  in  which  the 
combat  effectiveness  values  are  developed.  It  is  from  the 
scenarios  modeled,  the  force  structure  (both  Red  and  Blue) 
portrayed,  the  doctrine  used,  and  a  variety  of  assumptions  that 
the  combat  context  will  be  defined.  This  context  sets  the 
perspective  for  the  explicit  effectiveness.  It  is  important 
that  the  assumptions,  policies  and  terminology  used  are  con¬ 
sistent  with  the  decisionmaker's  view  of  the  world.  The 
decisionmaker's  confidence  in  the  programming  decisions 
resulting  from  the  VAA  approach  depends  entirely  on  his 
acceptance  of  the  combat  simulations  conducted. 

(2)  Another  important  aspect  of  combat  context  deals 
with  the  adequacy  of  a  model  to  analyze  the  alternatives 
associated  with  the  issue  being  investigated.  For  the  combat 
output  to  be  accepted,  the  models  used  must  be  capable  of 
investigating  the  alternatives  at  the  appropriate  level  of 
resolution.  The  need  to  have  the  appropriate  level  of  reso¬ 
lution  leads  to  the  requirement  for  multiple  models,  some  of 
which  will  be  designed  to  look  at  the  whole  battle  and  some 
whose  purpose  is  to  model  functional  areas.  In  order  to  conduct 
the  marginal  value  tradeoffs  inherent  in  the  VAA  methodology, 
all  of  the  alternatives  must  be  modeled  using  the  same  model  or 
equivalent  models.  This  problem  of  resolution  can  only  be 
solved  by  using  a  hierarchy  of  models.  A  linkage  between 
functional  area  models  of  high  resolution  and  larger  "force 
structure  models"  of  lesser  resolution  must  be  achieved.  The 
linkage  that  is  required  is  difficult  to  achieve  because  of 
problems  of  model  synchronization,  data  resolution,  and  battle 
dynamics . 

(3)  In  order  to  overcome  the  linkage  problem,  a  form  of 
calibration  must  be  accomplished  between  the  models  residing  at 
different  levels  of  the  hierarchy.  The  study  team  developed  a 
concept  of  hierarchical  MOEs  which,  when  used  in  conjunction 
with  standard  scenarios,  equivalent  data,  and  equivalent 
assumptions,  can  provide  the  linkage  necessary.  The  hierar¬ 
chical  MOEs  are  characterized  as  overarching  MOEs,  translational 
MOEs,  and  functional  MOEs.  These  MOEs  are  used  in  combination 
with  one  another  to  explore  an  alternative's  benefit  within  its 
functional  arena  while  relating  it  to  a  larger  context.  The 
overarching  MOEs  are  calculated  in  a  larger  context  "force 
structure  model"  and  are  used  to  measure  the  comprehensive 
"whole"  battle.  The  model  used  to  do  this  must  capture  all  of 
the  effects  of  the  systems  or  force  structure  alternatives  to  be 
investigated  and  will  probably  be  a  lower  resolution  model.  The 
output  for  each  alternative  will  vary  in  level  of  detail.  Some 
of  the  output  will  be  of  sufficient  detail  to  be  used  directly 
in  the  Value  Added  Analysis  without  further  high-resolution 
modeling.  However,  there  will  be  other  alternatives  for  which 
this  larger  ( low- resolution)  model  will  not  provide  sufficient 
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detail.  In  order  to  investigate  these  other  alternatives,  we 
must  find  a  way  to  link  the  low-resolution  model  with  higher- 
resolution  models  (usually  functional  area  models).  The 
translational  MOEs  are  the  link.  These  MOEs  can  be  measured  in 
both  the  low- resolution  and  the  higher-resolution  model.  The 
translational  MOE  is  used  to  transfer  the  results  of  the  low- 
resolution  model  (in  terms  of  sufficiency  criteria)  to  the 
higher-resolution  model.  The  functional  MOE  are  used  to  measure 
output  in  the  higher-resolution  model.  Figure  5-2  depicts  how 
models  and  these  MOEs  are  linked. 


Figure  5-2.  Combat  Contextual  Framework 

(4)  The  overarching  MOE  is  derived  from  a  model  which  is  used 
to  put  the  issue  in  the  largest  context  possible.  In  our  example, 
Vector-In-Command  (VIC)  or  Corps  Battle  Analyzer  (CORBAN)  is 
identified  as  the  highest  (in  terms  of  our  hierarchy)  level  model. 

The  overarching  MOEs  used  in  this  figure  are  COFM,  BFS,  and  MFCM. 

The  overarching  MOE  are  selected  based  on  two  criteria.  First,  does 
the  MOE  measure  parsunetric  changes  (input)  in  the  model  with 
sufficient  discrimination  to  conclude  that  the  change  was  the  cause 
of  the  new  MOE  value?  And  second,  can  we  a  priori  provide  a  level 
of  sufficiency  for  the  MOE?  The  concept  of  sufficiency  allows  for 
setting  goals  or  levels  of  value  which,  when  defined  by  the 
decisionmaker ( s ) ,  indicate  a  breakpoint  or  culminating  criteria. 

(5)  The  idea  of  sufficiency  does  not  necessarily  have  to  be 
an  a  priori  value.  There  are  occasions  when  the  use  of  the  higher- 
level  model  will  help  define  where  this  point  exists.  In  the  above 
figure,  the  parametric  dial  represents  the  ability  to  explore 
criterion  by  varying  input.  An  analyst  may  run  the  larger  context 
model  to  discover  the  breakpoints.  In  either  case,  the  purpose  for 
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using  the  Overarching  MOE  is  to  allow  the  analyst  and  decisionmaker 
to  identify  at  the  highest  contextual  level  a  point  at  which  an 
acceptable  solution  has  been  achieved.  Once  this  acceptable  solution 
has  been  reached,  we  can  determine  from  the  simulation  a  set  of 
translational  MOEs.  These  translational  MOEs  are  selected  because 
they  are  found  in  both  the  lower-resolution  model  (larger  context) 
and  in  the  higher-resolution  functional  model  (more  narrow  context). 
Because  these  translational  MOEs  exist  in  both  categories  of  models 
(the  Translational  MOE  is  a  measure  which  is  found  in  both  the 
higher-level  model  [VIC/CORBAN,  etc.]  and  the  lower-level  model 
[MACATK/COMO  III,  etc.]),  they  can  be  used  to  translate  sufficiency 
from  the  higher-level  model  to  the  lower-level  model.  The  ability  to 
translate  sufficiency  is  important  because  we  can  use  these 
translational  MOE  to  calibrate  across  many  different  functional 
models.  The  calibration  allows  us  to  create  equivalent  model  runs 
which  then  can  be  compared  against  one  another.  By  setting  all  of 
the  functional  models  to  equivalent  scenarios,  data,  and  levels  of 
performance  (sufficiency)  ,  we  believe  one  can  assume  that  a  10 
percent  change  in  one  model  is  equivalent  to  a  10  percent  change  in 
another.  Some  exeimple  translational  MOEs  may  be  loss  exchange 
ratios,  repair  return  rate,  system  exchange  ratios,  and  so  forth. 
Ultimately,  this  concept  is  useful  because  it  allows  the  analyst  to 
investigate  issues  in  their  "Native"  environment  by  the  use  of 
functional  models  and  reporting  the  results  using  functional  MOEs. 

(6)  The  ability  to  investigate  issues  within  a 
functional  context  is  important  because  it  allows  analysts  and 
decisionmakers  from  these  functional  areas  to  describe  value  in 
terms  they  understand  and  with  which  they  are  familiar.  It  also 
eliminates  the  need  to  force  a  result  into  an  MOE  which  may  not 
meet  the  need.  The  translational  MOE  provide  decisionmakers  at 
one  level  the  ability  to  communicate  in  a  very  structured  way 
their  view  of  the  need  or  level  of  sufficiency  to  decisionmakers 
and  analysts  at  another  level.  Furthermore,  this  ability  to  set 
criteria  at  a  level  which  accommodates  the  whole  allows  one  to 
set  criteria  in  many  different  functional  areas. 
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b.  Analytical  Medals  and  Tools 

(1)  Introduction.  As  part  of  the  methodology  devel¬ 
opment,  the  study  team  conducted  a  review  of  currently  available 
combat  models.  One  important  criterion  for  selection  of  the 
combat  model  for  the  VAA  Study  was  the  need  for  a  fast  running 
model.  In  fact,  the  team  had  initially  decided  to  use  a  highly 
aggregated  spreadsheet  combat  model  called  Corps  Model  (CORMO), 
developed  under  contract  to  PA&E  by  CACI,  Inc. -Federal.  CORMO 
was  selected  because  of  its  speed  and  ability  to  give  good  "ball 
park"  effectiveness  output.  However,  as  the  methodology 
matured,  it  became  apparent  that  selecting  the  resolution  of  the 
model  was  extremely  important.  This  insight  was  important 
enough  that  the  team  abandoned  its  original  model  choice  and 
developed  an  entirely  new  capability  that  would  simulate,  in 
detail,  a  weapon  system's  effect  on  a  large-scale  conflict. 

(a)  CORBAN  (the  Corps  Battle  Analyzer)  was  chosen  to 
simulate  the  large-scale  battle.  VAA  required  the  use  of  a 
corps-level  combat  model  to  help  determine  the  combat  effec¬ 
tiveness  value  added  to  the  corps  battle  by  various  weapon 
systems . 


(b)  This  focus  on  the  corps  level  is  important 
because  this  level  encompasses  a  fuller  mixture  of  maneuver, 
fire  support,  environment,  logistics,  and  command  and  control 
(C2)  than  other  levels  that  we  currently  model.  CORBAN,  as  a 
fast-running  corps-level  model,  has  the  sensitivity  necessary  to 
evaluate  weapon  systems  and  their  effect  on  the  corps  battle  in 
sufficient  detail  for  the  VAA  methodology. 

(2)  Model  Overview 

CORBAN  is  a  time-stepped,  closed-loop,  stochastic, 
combat  simulation  that  models  combat  at  the  corps /army  level. 

CORBAN  is  a  data  driven  model:  values  are  entered  by 
the  analyst,  not  fixed  by  the  code. 

CORBAN  can  simulate  a  range  of  battles  from  one 
battalion  versus  another,  to  eight  corps  versus  eight  Red 
armies.  Resolution  of  the  model  is  at  battalion  level. 

However,  individual  weapon  types  and  their  characteristics  are 
modeled . 


CORBAN  runs  on  the  VAX  family  of  computers.  The  model 
is  written  primarily  in  FORTRAN.  A  MIDAS  compiler  is  used  to 
compact  the  code  and  data.  The  model  consists  of  approximately 
30,000  lines  of  code  and  an  equal  aunount  of  input  data. 

While  running  on  a  VAX  8600,  the  model  executed  most 
VAA  runs  at  8  to  1  (simulation  hours  to  CPU  hours). 
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(3)  Input  Data.  Figure  5-3  depicts  the  CORBAN  data 
flow  as  used  for  the  Value  Added  Analysis  Study. 


CORBAN  DATAFLOW 
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Figure  5-3.  CORBAN  Data  Flow 

(a)  Scenario  and  Plans 

Task  organization.  CORBAN  allows  the  analyst  to 
task-organize  combat  formations  using  a  chain  of  command 
approach . 


Situation.  The  situation  defines  the  environment 
of  the  simulation  including  the  terrain  and  notes,  if  any. 
Missions  of  other  units  are  given  in  the  orders.  The  CORBAN 
battlefield  is  overlaid  with  a  nested-hexagonal  coordinate 
system  (each  large  hex  has  a  number  of  smaller  hexes  contained 
within).  This  nested-hexagon  coordinate  system  can  define  hex 
dieuneters  from  729  m  to  8,575  km.  Currently,  there  are  three 
levels  of  hexagons  defined,  at  the  25-km  level,  the  9-km  level, 
and  the  3.5-km  level  (size  is  defined  as  center  of  the  hex  to 
center  of  the  adjacent  hex).  Terrain  features  are  defined  only 
for  the  lowest  hexagon  level  (currently  the  3.5-km  level). 
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Terrain  within  a  hex  is  homogeneous  and  is  described  by  a 
combination  of  five  factors:  level  of  urbanization,  level  of 
forestation,  level  of  ruggedness,  the  quality  and  direction  of 
the  road  network  within  the  hex,  and  the  fordability  and 
location  of  rivers.  Effects  of  urbanization,  forestation,  and 
ruggedness  cover  the  entire  hex.  Roads  run  from  hex  center  to 
hex  center,  and  rivers  flow  along  the  borders  of  the  hex. 
Originally,  terrain  data  bases  were  developed  by  TRAC-FLVN  for 
the  CORDIVEM  Model  based  on  digitized  terrain.  This  data  was 
then  translated  into  CORBAN  terrain  data.  Currently,  Korea, 
Persian  Gulf,  and  Central  European  terrain  has  been  defined. 

Mission.  Both  sides  (Red  and  Blue)  are  given 
forces,  objectives,  contingencies,  and  time  constraints  under 
which  to  achieve  these  objectives. 

Execution.  Every  unit  is  given  one  or  more  orders, 
either  at  the  start  of  the  simulation  or  later  when  a  contin¬ 
gency  arises.  There  are  three  types  of  orders  in  CORBAN — 
simple,  complex,  and  templated.  The  simple  order  is  a  one-line 
entry  giving  a  mission  and  an  objective.  The  simple  order  is 
generally  used  for  separate  units  that  do  not  have  unique  phases 
or  subordinate  units.  A  complex  order  is  given  to  larger  units 
that  have  subordinate  units  and  that  require  unique  phasing  of 
operations.  Complex  orders  are  usually  given  to  Blue  divisions 
and  corps  and  to  Red  armies.  Templated  orders  are  usually  given 
to  Blue  battalions  and  brigades  and  to  Red  battalions,  brigades, 
and  divisions.  Each  templated  order  is  given  a  unique  desig¬ 
nator  that  can  be  passed  as  a  mission  by  means  of  a  simple  or 
complex  order.  Templated  orders  can  be  thought  of  as  a  unit 
standing  operating  procedure  (SOP). 

Service  Support.  CORBAN  models  several  aspects  of 
service  support.  First,  every  unit  is  given  ammunition  and  fuel 
amounts.  Then  assets  of  the  unit,  as  they  move  or  engage  in 
combat,  consume  the  unit's  supplies.  As  the  battle  progresses, 
requests  for  resupply  are  made,  and  logistical  units  dispatch 
convoys  to  resupply  the  combat  (fighting)  units.  Additionally, 
CORBAN  models  the  repair  of  assets  and  distribution  of  major  end 
items  to  units. 

Command  and  Signal.  To  execute  the  missions 
assigned,  CORBAN  uses  an  ingenious  command  and  control  system. 
Every  unit  on  the  battlefield  has  its  status  defined  with  a 
series  of  64  1-bit  flags.  When  certain  user-defined  events 
occur,  a  flag  for  the  unit  is  turned  on.  Based  upon  these 
flags,  the  mission  which  is  currently  being  executed  could 
change.  Superior  units  have  the  ability  to  change  their 
missions  based  upon  reading  subordinate  oostures.  This  allows  a 
commander  to  reassign  logistic  priorities,  inform  helicopter 
units  to  support  maneuver  units,  and  give  priorities  to 
artillery  support.  Units  in  CORBAN  can  pass  information  to  a 
commander,  a  subordinate,  or  to  all  units.  Through  the  command 
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and  control  system,  information  can  also  be  passed  globally  to 
all  units.  Red  or  Blue. 

(b)  TOE.  Units  consist  of  a  collection  of  equipment 
and  assets  given  to  the  unit  by  the  TOE.  These  TOEs  are  not 
like  regular  TOEs  knovm  by  most  people  associated  with  the  Army. 
Most  CORBAN  TOEs  represent  a  battalion  task  force  equipped  for 
combat.  Some  TOEs  represent  separate  companies  or  batteries  in 
order  to  portray  their  characteristics  more  accurately  e.g., 

MLRS  battalion.  Both  the  Blue  and  Red  sides  are  modeled  at 
these  levels  of  resolution.  The  modeler  has  the  capability  to 
portray  any  unit  at  any  echelon  determined  appropriate,  based  on 
other  input  data  available.  The  type  and  amount  of  equipment  in 
each  unit  is  scenario  dependent  and  can  be  different  for  each 
unit  (e.g.,  two  different  armor  task  forces  may  contain 
different  aunounts  and  types  of  equipment). 

(c)  Asset  Characteristics.  CORBAN  explicitly 
plays  the  majority  of  all  major  weapon  systems.  Additionally, 
fuel  and  ammunition  (including  tank,  artillery,  and  special 
weapons)  are  modeled.  Also,  support  assets  such  as  supply 
trucks  are  represented.  The  capabilities  of  every  asset  are 
defined  by  the  analyst.  These  capabilities  include: 

1.  SSPK  versus  12  target  types  over  4  range  bands 
and  8  f irer-target  postures  (e.g./  stationary  firer  versus 
stationary,  fully  exposed  target). 

2.  Type  of  ammunition  fired  versus  each  target 

type. 

3.  Fire  rate. 

4 .  Maximum  and  minimum  ranges . 

5.  Allocation  of  fire  among  target  types. 

6.  Acquisition  capability  of  the  onboard  sensors. 

7.  The  vulnerability  of  the  v/eapon  system  to 
different  ammunition  types. 

8.  Fuel  usage  and  loss  factors. 

9.  Ammunition  usage  and  loss  factors. 

10.  Crew-level  requirements. 

11.  RAM  (reliability,  availability,  and  maintain¬ 
ability)  factor 

Sensor  classes  are  used  by  assets  to  acquire  enemy  assets  during 
combat.  The  capability  of  the  sensor  is  defined  for  four 
ranges,  and  the  probability  of  detecting  a  target  asset  type  in 
various  postures,  for  both  day  and  night. 

(d)  Search  Patterns.  Search  patterns  represent  a 
unit's  ability  to  detect  enemy  units.  The  search  pattern  is 
used  when  attempting  to  evaluate  a  unit's  environment  (location 
of  enemy  units  for  movement  purposes)  and  to  build  target  lists. 
Additionally,  information  about  enemy  units  that  are  detected 
may  be  transferred  to  other  friendly  units. 
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(4)  Combat  Analysis  (Figure  5-4) 
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Figure  5-4.  Combat  Analysis  Process 

(a)  CORBAN  uses  a  time-stepped  and  simultaneous 
combat  process.  All  units  go  through  three  cycles  in  each  time 
interval.  These  three  cycles  are  shoot,  move,  and  recover. 
Analysts  can  set  the  time  interval.  The  VAA  CORBAN  Model 
interval  was  set  for  10  minutes. 

(b)  Most  of  the  processes  in  CORBAN  are  deter¬ 
ministic.  These  include  combat,  logistics,  fire  support,  and 
movement.  The  only  stochastic  process  in  CORBAN  is  perception. 
As  a  result,  the  detection  of  target  units  and  evaluation  of 
enemy  threats  is  stochastic. 

(c)  Shoot  cycle:  during  each  shoot  cycle,  every 
ground  unit  does  the  following — evaluates  its  own  strength, 
determines  threats  and  subsequent  force  ratios,  builds  target 
lists,  determines  target  priorities,  performs  combat,  calls  for 
fire  support,  and  requests  logistical  support.  HQ  units  assign 
fire  support  missions  to  supporting  units. 

(d)  Move  cycle:  during  each  move  cycle,  every  unit 
perceives  the  environment,  evaluates  its  own  strength,  selects  a 
movement  path,  and  moves  if  required.  Air  bases,  helicopter 
units,  and  logistic  units  create  sorties  and  convoys  and  move 
them. 
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(e)  Recover  cycle:  during  the  recover  cycle,  all 
units  apply  attrition  and  suppression  received  during  the  shoot 
cycle,  recover  from  effects  of  previous  suppression,  recon¬ 
stitute,  resupply,  and  maintain  units. 

(5)  Output 

(a)  The  CORBAN  Model  is  rich  in  the  output  that  it 
provides.  There  are  three  main  categories  of  output  that  the 
model  provides.  First,  through  "debug"  flags,  errors  in  data 
can  be  found  and  corrected.  Second,  various  types  of  simple 
measures  of  effectiveness  can  be  created  from  a  model  run. 
Finally,  CORBAN  has  a  number  of  postprocessing  progreims  that 
will  manipulate  output  data  to  look  at  more  complicated  measures 
of  effectiveness. 

(b)  The  MOE  that  CORBAN  analysts  most  commonly  use 

are : 


1.  Correlation  of  forces  and  means  (COFM):  this  is 
a  measure  of  the  Red  force  strength  in  an  attack  corridor  with 
respect  to  the  Blue  combat  strength  in  the  same  corridor. 

2.  Force  ratios  (FR);  this  is  a  measure  of  the 
overall  Red  forces  remaining  with  respect  to  the  overall  Blue 
forces  remaining. 

3.  Loss  exchange  ratios  (LER) :  this  is  a  measure 
of  the  combat  value  of  Red  systems  lost  to  the  combat  value  of 
the  Blue  combat  systems  lost. 

4.  Fractional  exchange  ratios  (FER):  this  is  a 
ratio  of  proportion  of  Red  systems  value  lost  to  the  proportion 
of  Blue  systems  value  lost. 

5.  System  effectiveness  ratios  (SER):  this  is  a 
ratio  of  the  number  of  Red  systems  killed  by  a  Blue  system  to 
the  number  of  Blue  systems  killed. 

6.  Killer-victim  tables:  these  tables  show  losses 
and  killers  by  asset  and  asset  category 

7.  Effective  battalions  remaining  (EBR);  this 
measure  evaluates  the  number  of  battalions  (generally  maneuver, 
artillery,  rocket,  and  helicopter)  remaining  on  each  side  that 
are  still  combat  effective. 

8.  Movement  of  force  center  of  mass  (MFCM) :  this 
measure  evaluates  the  performance  of  a  attacker  by  examining  the 
distance  the  center  of  mass  of  ail  of  his  forces  has  travelled. 

9.  Mission  accomplishment  (MA):  qualitative 
measure  of  whether  or  not  each  side's  missions  have  been 
accomplished . 
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10.  Front  line  trace:  this  qualitative  measure  is 

used  to  evaluate  such  items  as  potential,  or  real  breakthroughs, 
and  key  objectives  siezed  or  lost. 

11.  Plots  of  unit  locations:  allows  an  analyst  to 
evaluate  a  specific  unit's  performance  based  on  its  position 
relative  to  the  enemy  and  to  evaluate  a  large  formation's  (e.g., 
division)  intermediate,  or  final  dispositions. 

(6)  Debugging.  Much  of  the  information  for  the 
measures  of  effectiveness  comes  from  setting  certain  output 
flags  in  the  model.  Raw  data  that  can  be  quickly  output  from 
the  CORBAN  Model  includes: 

(a)  Decision  Trace.  This  information  informs  the 
analyst  when  a  unit  sets  a  situation  flag  and  when  a  unit 
responds  to  a  contingency  based  upon  those  flags.  It  also  tells 
an  analyst  what  mission  or  tactical  SOP  a  unit  is  executing. 
Finally,  it  gives  information  on  the  movement  pattern  and 
location  of  all  units  in  the  simulation. 

(b)  Communications  Trace.  This  file  informs  the 
analyst  when  information  is  sent  to  a  superior,  a  subordinate, 
or  to  ail  units. 

(c)  Unit  Status.  This  information,  gained  at 
intervals  throughout  the  run,  tells  the  analyst  where  all  units 
are,  what  they  are  doing,  and  their  posture  and  strength. 

(d)  Engagement  Trace.  Similar  to  the  decision 
trace,  this  file  records  all  engagements  by  time  step,  unit,  and 
location . 

5-3.  NONCOMBAT  EFFECTIVENESS 

a.  Introduction.  The  purpose  of  this  section  is  to 
illustrate  a  method  for  analyzing  a  noncombat  effectiveness 
issue  in  the  Army.  In  particular,  what  are  the  costs  and 
benefits  of  using  photovoltaic  (PV)  solar  systems  to  recharge 
batteries  versus  the  conventional  method  used  currently  for 
recharging  batteries  that  power  pop-up  targets  used  in  Army  tank 
training.  This  application  illustrates  the  issue  clarification, 
cost,  noncombat  effectiveness  (NCE)  and  optimization  modules  of 
the  Value  Added  methodology  (see  Figure  5-5). 
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Figure  5-5.  VAA  Methodology  -  Energy  Alternative 

Effectiveness 

b.  Photovoltaic  (PV)  Systems  Background.  Photo¬ 
voltaic  cells  convert  solar  energy  to  electricity  which  can  be 
used  in  many  different  tasks  such  as  battery  charging  and 
supplying  power  to  pocket  calculators.  There  are  numerous  PV 
applications  in  government  and  industry.  The  Tank  and 
Automotive  Command  (TACOM)  is  currently  testing  PV-powered 
trickle  chargers  for  use  in  trucks,  tanks,  and  armored  personnel 
carriers  which  have  long  periods  of  inactivity  between  uses. 
Communications  and  Electronics  Command  (CECOM)  is  investigating 
the  use  of  PV  systems  for  battery  charging  portable  communi¬ 
cation  sets.  PV  systems  have  the  following  beneficial  features; 
high  reliability,  modularity,  low  or  no  maintenance, 
nonpolluting,  silent  power,  and  no  fuel  or  water  requirements. 
Arthur  D.  Little,  Inc.,  under  contract  to  US  Army  Construction 
Engineering  Research  Laboratory  (CERL) ,  conducted  a  study  on 
potential  PV  applications  in  selected  Army  facilities.  The 
following  PV  the  applications  for  the  Army  were  cited  in  the 
CERL  study; 
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(1)  "A"  Stations  (measures  target  distances  for  testing 
weapon  systems) 

(2)  Battery  Chargers  (for  emergency  power  of  water 

wells ) 

(3)  Clearance  Lights 

(4)  Bugle  Recorders 

(5)  Global  Positioning  Systems  (GPS) 

(6)  Mobile  Firing  Ranges 

(7)  Mobile  Generators 

(8)  Radio  Repeaters 

(9)  Firing  Range  Guns 

(10)  Range  Surveillance  Videos 

(11)  Microwave  Towers 

(12)  Remote  Data  Acquisition 

(13)  Meteorological  Towers 

(14)  Storage  Facilities  (Igloos) 

(15)  Microwave  Repeaters 

(16)  Pop-up  Targets 

c.  Issue  Clarification.  The  issue  selected  as  the 
noncombat  effectiveness  case  study  was  whether  to  acqui 
photovoltaic  panels  or  to  continue  using  conventional  meuns  to 
recharge  batteries  that  power  radio  operated  pop-up  targets  used 
in  tank  training  Army-wide,  Table  5-1  gives  an  estimate  of  the 
quantity  of  radio  operated  pop-up  targets  powered  by  batteries 
in  the  Army.  According  to  the  Army  Training  Support  Center, 
there  are  batteries  at  the  3,320  battery-powered  pop-up  targets 
used  for  tank  training  in  the  Army  (not  including  USAREUR).  Pop¬ 
up  targets  exist  in  USAREUR,  but  data  on  the  quantity  is  not 
currently  available.  The  specific  issue  addresses  whether  the 
batteries  at  the  3,320  radio  controlled  pop-up  targets  should 
remain  battery  powered  continue  to  be  recharged  at  battery  shop 
facilities  (conventional  method)  or  should  be  recharged  by  on¬ 
site  PV  panels.  The  noncombat  measure  of  effectiveness  applied 
in  this  case  study  is  energy  required  to  operate  the  pop-up 
targets . 
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Table  5-1.  Estimated  Number  of  Pop-up  Targets  in  the  Army 

(Excludes  USAREUR) 


Battery  Operated  Radio  Controlled 

Pop-up  Targets 

Armor 

8th  Army 

73 

FORSCOM 

993 

Total 

1066 

1  Infantry 

8th  Army 

199 

FORSCOM 

1668 

WESTCOM 

387 

Total 

2254 

Armor  and  Infantry  Total 

3320 

Source:  Training  Support  Center,  Ft  Eustis. 

d.  DOD  and  Army  Energy  Policy 

(1)  DOD  Energy  Policy.  Figure  5-6  displays  Title  10, 
US  Code  Section  2394  guidance  for  DOD  energy  policy.  The  policy 
requires  procurement  of  renewable  forms  of  energy  (e.g.,  PV) 
when  it  is  cost  effective,  provides  guidelines  for  the 
computation  of  the  life  cycle  costs  of  renewable  energy  systems, 
and  mandates  reports  to  Congress  by  the  Secretary  of  Defense 
within  two  year  intervals,  of  the  results  of  studies  conducted 
with  respect  to  the  use  of  renewable  forms  of  energy  in 
supplying  the  energy  needs  of  DOD.  Cost  effectiveness  refers  to 
the  lowest  cost  energy  source  that  can  meet  a  given  set  of 
energy  requirements  in  support  of  specified  mission  objectives. 
The  pop-up  target  case  study  presented  here  serves  as  a  simple 
excimple  of  the  kind  of  analysis  that  could  be  conducted  in 
support  of  DOD  energy  policy. 
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Title  10,  U.S.  Code  Section  2394 

2394a.  Procurement  of  energy  systems  using  renewable  forms  of  energy 

(a)  In  procuring  energy  systems  the  Secretary  of  a  military  department  shall  procure  systems  that  use  solar  enetg\-  nr 
other  renewable  forms  of  energy  whenever  the  Secretary  determines  that  such  procurement  is  possible  and  will  he  cost  efn- 
nve.  reliable,  and  otherwise  suited  to  supplying  the  energy  needs  of  the  military  department  under  his  jurisdicnon 

(b/l)  The  Secretary  of  Defense  shall  from  time  to  nme  studv  uses  for  solar  energy  and  other  renewable  forms  of  energy  in 
determine  what  uses  of  such  fomis  of  energy  may  be  cost  etjeaive  and  reliable  in  .supplying  ihe  energy  needs  of  the  Depart- 
ment  of  Defense.  The  Secretary  of  Defen.se.  based  upon  the  results  of  such  studies,  shall  from  time  to  time  issue  policy  guide¬ 
lines  to  be followed  by  the  Secretaries  of  the  military  departments  in  carrying  out  subsection  (a)  and  section  285  7  of  this  title 

(2!  The  Secretary  of  Defense  shall  submit  to  the  Committees  on  Armed  Services  of  the  Senate  and' House  of  Representa¬ 
tives  not  less  often  than  every  two  years  a  report  on  the  studies  conduaed  pursuant  to  paragraph  (1).  Each  .such  report  shall 
include  any  findings  of  the  Secretary  with  respea  to  the  use  of  solar  energy  and  other  renewable  forms  of  energy  in  .supplying 
the  ener^  needs  of  the  Department  of  Defense  and  any  recommendations  of  the  Secretary  for  changes  In  law  that  mm  he 
appropriate  in  light  of  such  studies. 

feXD  For  the  purposes  of  this  section,  an  energy-  system  using  solar  energy  or  other  renewable  forms  of  energy  shall  he 
considered  to  be  cost  Receive  tf  the  difference  between  (A)  the  anginal  investment  cost  of  the  energy  system  using  such  a  form 
of  energ/.  and  (B)  the  original  investment  cost  of  the  energy  sv.stem  not  using  such  a  form  of  ener^  can  be  recovered  over  the 
expected  life  of  the  system. 

Q)  A  determination  under  paragraph  (1)  of  whether  a  cost-differential  can  be  recovered  over  the  expeaed  life  of  a  ‘■ystem 
shall  be  made  using  accept^  life-cycle  costing  procedures  and  shall  include  ■ 

lA)  the  use  of  all  capital  expenses  and  all  operating  and  maintenance  expenses  associated  with  the  energy  system  us¬ 
ing  solar  energy  or  other  renewable  forms  of  energy,  and  not  using  such  a  form  of  energy,  over  the  expeaed  life  of  the 
system  or  during  a  period  of  25  yeon  whichever  is  shorter: 

(B)  the  use  of fossil  fuel  costs  (and  a  rate  of  cost  growth  for fossil  fuel  costs)  as  determined  by  the  Secretary  of  Offense, 
and  s 

tC)  the  use  of  a  discount  rate  of  7  percent  per  year  for  all  expenses  of  the  energy  system. 

(3)  For  the  purpose  of  any  life-cycle  cost  analysis  under  th-x  suhseaion.  the  anginal  investment  cost  of  the  energ  i  sTem 
using  solar  energy  or  other  renewable  forms  of  energy  shall  he  reduced  by  10  percent  to  reflect  an  allowance  tor 
an  investment  cost  credit. 

(Added  Pub.  L  97-321.  title  VIII  SOKaXl)  Oa.  15.  1982.  96  Stat.  1569.  and  amended  Pub.  L  98-525.  tule  XIV  1 -1051361. 
Oa.  19.  1984.  98  Stat.  2623.) 


Figure  5-6.  Title  10  US  Code,  Section  2394  Guidance 
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(2)  Army  Energy  Policy  Objectives.  The  Army  Energy 
Office  in  ODCSLOG  presents  the  Army's  current  energy  goals  in 
the  Army  Energy  Resources  Management  Plan  (FY  86-95).  The  Plan 
includes  goals  regarding  the  Army's  use  of  renewable  energy 
sources — sunlight,  wind,  geothermal,  hydropower,  ocean  thermal, 
and  biomass.  In  particular,  the  Army  aims  to  increase  its  use 
of  renewable  energy  technologies  between  FY  86  and  95  by  5 
percent  or  27,975  MBtu  (millions  of  British  Thermal  units). 

This  estimate  is  based  upon  FY  1985  renewable  energy  use  of 
559,510  MBtu  in  the  Army.  The  amount  of  energy  required  to 
power  the  pop-up  targets  in  this  time  period  (1985-1995)  is 
estimated  to  be  22,180  MBtu.^  If  this  energy  requirement  were 
"cost  effectively"  met  by  PV  technologies,  then  about  80  percent 
of  the  Army's  renewable  energy  objectives  would  be  satisfied. 
Although  not  addressed  in  this  report,  a  more  complex  case  study 
would  compare  the  economics  of  conventionally  recharged 
batteries  against  all  technically  applicable  renewable 
technologies  -  not  just  photovoltaics .  This  kind  of  analysis 
should  consider  both  pop-up  target  energy  requirements  and  the 
change  in  the  Army's  renewable  energy  usage  as  noncombat  MOE. 
However,  because  of  the  Army's  success  with  photovoltaic 
applications  and  to  simply  illustrate  the  concept,  the 
discussion  will  be  limited  to  photovoltaics. 

e.  Cost-Benefit  Analysis.  The  issue  of  whether  to 
continue  recharging  batteries  conventionally  or  to  invest  in 
photovoltaics  for  pop-up  targets  requires  identification  of  the 
relative  costs  and  benefits.  The  focus  of  this  case  study  will 
be  on  monetary  costs  and  benefits  (cost  savings)  as  stipulated 
by  Title  10,  U.S.  Code  Section  2394.  The  investment  criterion 
applied  (as  required  by  Title  10)  is  the  number  of  years  that 
the  photovoltaic  alternative  "pays  back"  (in  discounted 
dollars )  . 


(1)  Cost  Assumptions.  The  following  assumptions 
were  used  in  estimating  Life  Cycle  Costs  (LCC)  for  both  the 
conventional  and  PV  methods  for  recharging  batteries  used  to 
power  pop-up  targets: 

Economic  useful  life  of  a  PV  system:  20  years^ 
Economic  useful  life  of  a  battery;  2  years^ 


^ Cased  on;  Applications  Survey  for  Remote  Photovoltaic  Power  Systems,  Arthur  D. 
Little,  Inc.,  October  1989,  Table  1.  (Estimated  energy  savings  from  PV  recharging: 
3320  batteries  x  480  WH/day  x  120  days  =  191.2  MWH  x  11.6  BTU/WH  =  2218  MBTU). 
^Source:  Life  Cycle  Cost  Analysis  of  Photovoltaic  Power  Systems,  U.S.  Army 
Engineering  and  Housing  Support  Center.  (Engineering  and  Housing  Support 
Center). 

^Average  figure  for  operating  training  center  ranges  and  life  of  battery  based  on 
discussions  with  ODCSOPS  (DAMO-TR)  and  CERL. 
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Economic  useful  life  of  a  battery  with  PV:  5  years! 
Pop-up  targets  are  used  120  days  a  year  operating  at 
10  hours  a  day.! '2 

Batteries  are  recharged  after  each  training  exercise 
(25  miles  round  trip  to  pick  up  and  return  battery 
after  recharging  -  one  training  exercise  per  day) 

A  fully  discharged  55  Amphour  (Ah)  12  Volt  battery 
requires  5 5 Ah  x  12  Volts  =  .66  KILOWATT  (kWh)  of 
electrical  energy  to  return  a  full  state  of  charge^ 
Cost  per  kWh  =  $.05^ 

Batteries  are  replaced  every  other  year! 

Battery  loss  rate;  5  percent! 

Nominal  Discount  Rate  (NDR):  7  percent^ 

Annual  inflation  factor:  4  percent 

(2)  Conventional  Battery  Costs.  A  12 -volt 
battery,  replaced  every  other  year,  is  the  conventional  energy 
source  used  to  power  the  pop-up  targets  specified.  Battery  costs 
were  based  on  information  obtained  from  the  National  Training 
Center  (NTC)  located  at  Ft  Irwin,  California  and  from  CERL,  The 
battery  procurement  cost  is  approximately  $100.  Assuming  that 
all  3320  battery-powered  pop-up  targets  in  the  Army  are  utilized 
120  days  per  year  and  the  batteries  are  replaced  every  other 
year  (with  a  5%  loss  rate  incurred  in  transportation)  then  1826 
batteries  should  be  procured  annually  (i.e.  1660  batteries  plus 
3320  batteries  x  .05  =  1826  batteries  replaced  annually).  Based 
upon  this  procurement,  the  annual  cost  for  pop-up  target  battery 
replacement  is  $182,600  ($100  x  1826  batteries).  O&M  costs  over 
20  years  were  estimated  to  be  $2.65M  annually.  This  includes 
$2.39  for  transportation  costs  (annual  transportation  costs  = 
3320  batteries  x  120  rechargings  per  year  x  25  miles  x  $.24  per 
mile)"!  and  $.013M  for  electrical  energy  costs  (annual  electrical 
energy  costs  =  3320  batteries  x  120  rechargings  per  x 
. 66kWh/battery  x  $.05/kWh).2  The  $.24  per  mile  O&M  cost  factor 
excludes  manpower  costs  for  the  two  man  crew  used  to  transport 
and  install  50  batteries  per  10  hour  day.  (With  3320  battery 
recharges  estimated  daily  for  120  training  days,  796,800  man¬ 
hours  would  be  expended  annually  for  battery  recharging) .  The 
present  values  for  battery  procurement  and  O&M  costs,  over  20 


^Discussions  with  CERL  personnel. 

“Based  on;  Applications  Survey  for  Remote  Photovoltaic  Power  Systems,  Arthur  D. 
Little,  Inc.,  October  1989,  Table  1.  (Estimated  energy  savings  from  PV  recharging; 
3320  batteries  x  480  WH/day  x  120  days  =  191.2  MWH  x  11.6  BTU/WH  =  2218  MBTL').'' 
^Title  10,  U.S.  Code  Section  2394.  Guidance  includes  use  of  7  percent  discount  rate 
for  all  expenses  of  energy  systems. 

■^Source;  Life  Cycle  Cost  Analysis  of  Photovoltaic  Power  Systems,  U.S.  Army 
Engineering  and  Housing  Support  Center.  (Engineering  and  Housing  Support 
Center). 


5-18 


CAA-SR-91-9 


years,  amounted  to  $1.93M  and  $34,55M,  respectively^ .  The 
present  value  of  conventional  battery  life  cycle  costs  for  the 
20  year  period  is  $37.4M. 

(3)  PV  Costs.  PV  investment  costs  per  pop-up 
target  were  estimated  at  $300  by  CERL.  The  3320  operating  pop¬ 
up  targets  in  CONUS  require  3320  PV  systems.  Thus,  the  total 
initial  investment  for  PV  amounts  to  $.996M  (3320  batteries  x 
$300.00)  with  an  annual  recurring  cost  for  operation  and 
maintenance  at  7.5  percent  of  investment  cost  or  about  $74,500 
per  year. 2.  since  battery  life  is  extended  to  5  years  and  there 
is  no  requirement  to  transport  batteries  with  PV  systems, 
battery  costs  are  reduced  to  $66,400  annually  (1/5  x  3320 
batteries  x  $100  per  battery).  The  present  value  (over  20  years) 
for  the  initial  PV  investment  is  $.996M,  $1.08M  for  PV  O&M  costs 
and  $.96M  for  battery  replacement  costs.  Total  present  value 
costs  for  the  PV  alternative  is  $3.04  million. 

f.  Cost  Benefit  Results.  The  use  of  PV  as  an 
energy  source  to  recharge  batteries  for  pop-up  targets  was  found 
to  be  more  cost  beneficial  when  compared  to  the  use  of 
conventional  batteries  as  a  power  source.  Over  the  20-year 
economic  useful  life  of  a  PV  system,  total  present  value  cost 
savings  from  PV  investment  was  estimated  to  be  $37 . 4M-$3 . 04M  = 
$34.36M.  This  amounts  to  an  average  annual  rate  of  return  of 
56.5  percent.  The  discounted  payback  for  an  initial  investment 
of  $.996M  would  be  less  than  2  years. 

g.  Cost  Benefit  Finding.  Title  10,  US  Code, 

Section  2394,  indicates  that  renewable  energy  investment  with  a 
payback  within  its  economic  life  (20  years)  is  considered  to  be 
cost  effective  and  should  be  undertaken.  The  PV  case  study 
described  in  this  chapter  pays  back  in  less  than  2  years . 
Furthermore,  it  ha?  been  indicated  that  the  PV  investment 
specified  significantly  contributes  to  the  Army's  renewable 
energy  objectives.  Additional  benefits  would  be  realized  by  PV 
investment,  such  as  its  contribution  to  Army  environmental  goals 
(such  as  less  handling  and  disposing  of  batteries),  but  were  not 
quantified  in  this  study.  The  execution  of  the  remaining 
modules  in  the  VAA  methodology  (e.g.,  implicit  factors)  was  not 
considered  necessary  for  this  particular  issue,  since  the  PV 
investment  described  met  the  cost  effectiveness  criterion 
provided  in  Title  10. 


5-4.  SUMMARY.  The  development  of  the  Value  Added  methodology 
specifically  included  two  components  (Explicit  and  Implicit)  to 
the  valuation  portion  of  VAA.  The  explicit  effectiveness 


^Title  10,  U.S.  Code  Section  2394.  Guidance  includes  use  of  7  percent  discount  rate 
for  all  expenses  of  energy  systems. 

“Source  for  cost  factors:  Photovoltaics  for  Military  Applications,  Meridian 
Corporation,  May  1988. 
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investment  described  met  the  cost  effectiveness  criterion 
provided  in  Title  10. 


5-4.  SUMMARY.  The  development  of  the  Value  Added  methodology 
specifically  included  two  components  (Explicit  and  Implicit)  to 
the  valuation  portion  of  VAA.  The  explicit  effectiveness 
component  was  specifically  designed  to  measure  the  quantifiable 
element  of  value.  The  methodology,  as  currently  envisioned, 
uses  traditional  analysis  tools  such  as  combat  simulations, 
historical  data,  or  mathematical  formula  to  measure  the  explicit 
contribution  an  alternative  would  provide  to  the  Total  Army 
Program.  This  explicit  value  is  always  measured  from  a  baseline 
position,  e.g.,  in  our  illustrative  example,  we  used  the  FY  94 
Progreun  Force.  The  module  appears  to  be  flexible  enough  to 
handle  a  wide  range  of  quantification  models  and  methods.  The 
results  of  this  module  are  passed  to  the  Effectiveness 
Integration  Module  as  changes  from  the  baseline  position. 
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CHAPTER  6 

VALUE  ADDED  COSTING 


6-1.  INTRODUCTION 

a.  The  purpose  of  this  chapter  is  to  provide  a  discussion 
of  the  Costing  Module  of  the  Value  Added  methodology.  Included 
in  this  discussion  are  the  methods  and  processes  selected  and 
their  relationship  to  the  optimization  module.  The  Value  Added 
methodology  required  a  breakdown  of  life  cycle  weapon  system(s) 
cost  elements  coupled  with  a  mechanism  for  adjusting  costs  to 
account  for  variations  in  procurement  quantities.  Consequently, 
two  PC-based  models  were  developed — the  Cost  Quantity  Model 
(CQM)  and  the  Weapon  System  Life  Cycle  Cost  Model  (LCCM).  In 
addition  to  the  two  PC-based  models,  the  Costing  Module  uses  a 
methodology  for  developing  weapon  system  definitions  based  on 
the  Army  Resources  Integration  Management  (ARIM)  Study.  This 
chapter  is  structured  as  follows: 

(1)  Introduction 

(2)  Costing  Context 

(3)  Select  or  Develop  Models  and  Tools 

(4)  Implement  Analytical  Models  and  Tools 

(5)  Summary  and  Observations 

b.  Figure  6-1  depicts  the  placement  of  the  Costing  Module  in  the 
overall  Value  Added  methodology.  The  majority  of  the  inputs  for  this 
module  are  fed  to  it  from  the  Issue  Clarification  Module.  The 
outputs  generated  in  the  costing  module  are  passed  to  the 
optimization  module  for  the  conduct  of  a  cost  benefit  analysis. 
Costing  output  also  can  be  passed  to  the  Results  and  Display  Module 
for  direct  analysis  and  presentation. 
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Figure  6-1. 
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c.  Figure  6-2  provides  a  flow  chart  of  the  Costing  Module. 


FLYAWAY  COST 


WEAPON  SYSTEM  COST 


PROCUREMENT  COST 


PROGRAM  ACQUISITION  COST 


LIFE  CYCLE  COST 


Data 

Sources 

3C£ 

PROBE 

ACP 

Other 

Figure  6-2.  Costing  Module  Flow  Chart 
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6-2.  DEVELOPMENT  OF  THE  COSTING  CONTEXT 

a.  Identify  Assumptions 

(1)  Experience  curves  for  weapon  systems  are  computed  at 
the  component  level.  Models  rolling  up  component- level  cost  and 
quantity  data  reside  with  the  program  manager  of  each  system. 

The  Value  Added  Analysis  methodology  may  address  several  weapon 
systems  simultaneously,  making  use  of  component-driven  models 
impractical  in  a  quick  reaction  environment. 

(2)  Regression  analysis  of  historical  or  projected 
annual  weapon  system  production  costs  can  be  used  to  calculate 
system  first  unit  costs  and  experience  and  production  rate 
curves  that  generate  average  annual  production  cost  within  an 
acceptable  margin  of  error. 

(3)  Similarly,  P92  cost  codes  for  the  five  major  cost 
categories  (development,  production,  military  construction, 
fielding,  and  sustainment)  can  be  prorated  with  the  impact  on 
the  affected  appropriation  category  based  on  experience/ 
production  rate  curves  and  cost  factors  generated  from  available 
data. 

b.  Identify  Cost  Policy.  Army  weapon  system  costing 
policy  is  addressed  in  Army  Regulation  (AR)  70-2  (Integration  of 
Weapon  Systems  Costing  Programing  and  Execution  Management 
Systems,  June  1986).  CEAC  is  the  proponent  for  Army  weapon 
system  costing.  Army  inflation  guidance  is  provided  every  2 
years  by  the  Army  Materiel  Command  (AMC). 

c.  Determine  Lexicon.  Definitions  for  the  five 
categories  of  weapon  system  cost  were  identified  as:  Fly  Away, 
Procurement,  Weapon  System,  Program  Acquisition,  and  Total  Life 
Cycle  Costs.  Procurement /production  costs  were  selected  as  the 
basis  for  optimizing  system  procurement.  The  P92  cost  elements 
as  defined  by  CEAC  in  DCA  -P92(r)  report  were  used  as  a  point  of 
departure  for  identifying  costs  elements  for  a  weapon  system. The 
Army  Resource  Integration  and  Management  (ARIM)  Study  addressed 
the  problem  of  consistent  weapon  system  definition.  As  part  of 
that  effort,  the  costs  for  the  five  major  weapon  system  descrip¬ 
tions  were  identified.  These  five  major  types  of  weapon  system 
costs  are:  Fly  Away,  Weapon  System,  Procurement,  Program 
Acquisition,  and  Life  Cycle  as  shown  in  Figure  6-3  with  the 
respective  cost  compositions. 
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Figure  6-3.  Life  Cycle  Cost  Composition 
6-3.  DETERMINE  APPRORIATE  MODELS  AND  TOOLS 
a.  Identify  Selection  Criteria 

(1)  Models  and  analytical  tools  used  in  the  Value  Added 
Analysis  methodology  for  costing  were  selected  based  on  the 
following  criteria; 

(a)  The  ability  to  perform  quick  reaction  weapon 
system  cost  estimates  considering  variation  in  procurement 
quantities . 

(b)  The  ability  to  provide  consistency  among  all 
weapon  systems  in  the  computation  of  cost/quantity 
relationships . 

(c)  The  ability  to  estimate  broad  cost  categories  and 
their  relationship  to  Army  appropriations  that  are  of  interest 
to  budget/programing  decisionmakers. 

(2)  The  study  team  conducted  a  subjective  analysis  of 
the  candidate  technique  using  the  above  criteria.  This  analysis 
consisted  of: 

(a)  Determining  analytical  tools  that  can  provide 
same  day  cost  estimates. 

(b)  Determining  a  consistent  cost  format  for  all 
weapon  systems . 
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(c)  Providing  the  capability  to  cross-reference 
between  weapon  system  cost  elements  and  Army  appropriations. 

b.  Determine  Measures  of  Effectiveness.  The 

following  MOE  are  relevant  for  the  Value  Added  Analysis 
methodology  model/tool  selection: 

(1)  Timeliness.  Costing  for  the  Value  Added  Analysis 
methodology  requires  extremely  quick  reaction  turnaround  because 
the  environment  in  which  these  types  of  decisions  are  conducted 
is  characterized  by  short  timelines. 

(2)  Accuracy.  Cost  estimates  must  be  accurate  within 
an  acceptable  margin  of  error  to  be  of  any  value  in  the 
analysis.  A  tradeoff  exists  between  the  timeliness  and  accuracy 
of  MOEs.  Given  the  tradeoff  with  timeliness  and  the  anticipated 
broad  scope  of  Value  Added  Analysis  applications,  selection  of 
models/tools  were  made  with  an  implicit  goal  of  providing  cost 
estimates  with  less  than  a  15  percent  margin  of  error. 

c.  Review  Alternative  Costing  Methods 
(1)  Cost  Quantity  Alternatives 

(a)  Program  Manager  Level  Models.  Production 
costs  of  a  weapon  system  are  conventionally  estimated  by 
aggregating  the  costs  of  the  individual  components.  The  costs 
of  the  components  are  calculated  using  learning  curves,  the 
required  quantities,  and  the  first  unit  costs  of  each  component. 
This  method  requires  considerable  computation,  especially  wnen 
the  number  of  components  is  large.  Models  performing  these 
"component"  aggregation  cost  estimates  are  complex  and  usually 
reside  at  the  progreun  manager  level. 

(b)  Composite  Learning  Curves.  An  estimation 
procedure  was  developed  aimed  at  reducing  the  computational 
effort  required  to  aggregate  component- level  costs  while 
yielding  reasonably  accurate  weapon  system  costs.  This 
procedure  replaces  individual  component-level  learning  curves 
with  a  composite  learning  curve.  It  allowed  for  the  calculation 
of  production  costs  with  simplified  computations  but  with  a  loss 
of  some  accuracy.  However,  tne  laborious  task  of  gathering 
component-level  learning  curve  data  exits  with  this  method. 

(2)  The  CQM  Method.  The  CQM  utilizes  experience  curve 
theory  to  estimate  average  unit  production  costs.  Base  experi¬ 
ence  curve  theory  specifies  unit  cost  as  a  function  of  the 
quantity  produced.  The  tenet  is  that  as  production  of  a 
particular  good  (weapon  system)  doubles,  production  cost 
decreases  at  some  constant  rate.  The  base  experience  curve 
method  is  an  accepted  technique  to  forecast  production  costs 
from  the  known  costs  of  past  production.  It  is  specified  as 
follows : 
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[6.01] 


y-AA^" 


where : 

y  =  average  unit  cost 
A =  cost  of  the  first  unit  produced 
X =  cumulative  quantity  of  units  produced 
b=  experience  curve  slope  coefficient. 


(a)  The  CQM  employs  an  altered  form  of  the  base 
experience  curve.  This  is  due  to  the  wide  recognition  that  the 
production  rate  plays  an  important  role  in  production  costs.  In 
light  of  this,  a  production  rate  variable  has  been  added  to  the 
base  method.  This  reveals  the  following  experience  curve 
function,  which  is  utilized  by  the  CQM.^ 


y  -  AQ‘’R‘ 


[6.02] 


where : 

Q -  cumulative  lot  mid-point 

b=  cost/lot  mid-point  slope  coefficient 
R =  production  rate 
c-  cost/rate  slope  coefficient 

All  other  variables  are  as  previously  defined. 

1.  In  addition  to  the  inclusion  of  the  production 
rate  variable,  [6.02]  above  has  been  altered  in  another  way. 

The  cumulative  quantity  variable  {X)  has  been  substituted  by  the 
cumulative  lot  mid-point  (0).  This  is  necessary  because  weapon 
system! s)  program  managers  generally  do  not  keep  cost  data  on 
individual  units  of  output.  Instead,  they  maintain  cost  data  on 
"lots"  of  output.  Since  the  cost  of  individual  units  is 
unknown,  estimation  of  the  experience  curve  as  specified  by  the 
base  method  is  impossible. 

Z-  One  method  to  remedy  this  is  use  of  the 
cumulative  lot  mid-point.  The  lot  mid-point  is  that  unit  in  a 
given  lot  of  output  whose  cost  is  equal  to  the  average  cost  of 
all  the  units  in  the  lot.  This  can  be  seen  graphically  in 


'See  Bemis.  John  C.  The  Cost  Implications  of  Production  Rate.  Concepts,  Spring 
1981,  VoluriK  i.  Number  2.  Pgs  84-94. 
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Figure  6-4.  Unit  cost  is  plotted  along  the  vertical  axis,  while 
the  number  of  units  produced  lies  along  the  horizontal  axis. 
Three  separate  lots  of  output  are  drawn  in  the  figure.  The 
plots  marked  by  Qi,  Q2,  and  Q2  represent  those  units  whose  cost 
equals  the  average  cost  of  the  entire  lot.  In  the  figure,  a 
curve  has  been  drawn  through  the  respective  lot  mid-points . 

This  method  is  helpful  because  weapon  system(s) 
program  managers  do  maintain  data  on  the  number  of  units 
produced  in  a  given  lot.  Coupling  this  with  the  cost  of  that 
lot  allows  a  cost/quantity  relationship  to  be  established.  This 
allows  the  experience  curve  to  be  estimated.  The  estimator  for 
the  cumulative  lot  mid-point  is  presented  on  the  next  page. 2 


[6.03] 


Q- 


F  +  L-*-24fL 

4 


Figure  6-4.  Lot  Mid-Point 


where , 

approximate  cumulative  lot  mid-point 
f=  first  unit  in  the  lot 
I  =  last  unit  in  the  lot 

Table  F-2  in  Appendix  F  lists  excimples  of  cumulative  lot  mid¬ 
point  calculations. 


“  Cost  Estimating  for  Engineers.  ALM-63-3126-LC.  Pgs  210-212. 
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ji.  Once  all  the  necessary  data  is  acquired,  the 
experience  curve  function  of  equation  [6.02]  can  be  estimated. 
This  is  done  by  applying  ordinary  least  squares  regression  to 
equation  [6.02].  In  order  to  do  so,  [6.02]  must  first  be  made 
log-linear.  Ordinary  least  squares  is  then  applied  to  the  log- 
transformed  variables  of  [6.02].  This  is  all  done  by  the  CQM. 
The  only  user  requirement  is  entry  of  the  variable  data. 

5.  Output  from  CQM  will  include  experience  curve 
estimates  and  descriptive  statistics.  In  cases  where  the 
production  rate  variable  is  found  to  be  statistically 
insignificant,  CQM  allows  the  user  to  drop  that  variable  and  re- 
estimate  the  curve.  Included  in  the  output  are  the  experience 
curve  percentages.  These  reveal  the  percentage  of  the  first 
unit  cost  to  which  first  unit  cost  decreases  when  production 
quantity  doubles.  Refer  to  Appendix  F  for  further  explanation. 

6,.  Total  production  costs  for  a  given  weapon  system 
are  found  by  finding  the  area  under  the  experience  curve  (see 
Appendix  F,  paragraph  F-6).  The  average  unit  cost  for  the 
projected  production  quantity  for  each  affected  year  is 
calculated  from  the  estimated  first  unit  cost,  experience  curve, 
and  production  rate  curve.  This  average  unit  cost  for  the 
affected  years  is  then  used  as  input  to  FOMOA,  which  computes 
the  optimal  procurement  quantities.  After  the  new  procurement 
quantities  are  determined  the  CQM  should  be  run  one  more 
iteration  to  compute  the  average  unit  costs  (by  year)  for  the 
optimal  production  quantities  generated  by  FOMOA. The  use  of  a 
constant  average  unit  cost  (AUC)  for  production  did  not,  for 
this  demonstration,  create  a  s.»_.-ation  in  which  a  stable 
solution  was  unattainable.  However,  the  danger  does  exist  that 
with  repeated  iterations  convergence  to  a  stable  solution  may 
not  be  obtainable.  Further  work  in  regards  to  dynamic  costing 
within  the  linear  program  must  be  conducted  as  part  of  the  VAA 
Phase  II  study. 

(3)  BCE  Costing  Updating 

(a)  The  BCE  is  a  generic  term  denoting  a  compre¬ 
hensive,  detailed  and  fully  documented  estimate  of  materiel  life 
cycle  costs  accomplished  by  the  weapon  system  project  manager. 
The  BCE  is  a  dynamic  document,  refined  and  updated  throughout 
the  acquisition  cycle.  The  BCE  serves,  after  review  and 
validation,  as  the  principal  cost  estimate  for  the  system  and  is 
used  as: 


1.  The  principal  institutional  source  document  for 
cost  information  related  to  materiel  systems. 

2.  The  basis  for  projecting  funding  requirements 
for  acquisition  and  operation  of  the  materiel  system. 
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2.  The  benchmark  (initial  BCE)  and  source  (updated 
BCEs )  for  system  cost  tracking. 

4.  The  basis  for  cost  inputs  to  such  reports  as  the 
Selected  Acquisition  Report  (SAR). 

(b)  It  would  be  impractical  to  update  BCEs  to  address 
issues  associated  with  Value  Added  Analysis  applications. 
Therefore,  a  suitable  strategy  to  estimate  key  cost  elements 
comprising  total  life  cycle  cost  of  a  weapon  system  was 
undertaken. 

(4)  The  LCCM  Method.  Once  the  optimal  procurement 
quantities  and  production  costs  are  calculated,  the  LCCM 
redistributes  production,  fielding  and  sustainment  costs.  This 
is  accomplished  by  inputing  cost  factors  generated  from  the 
BCE/ACP  source  data  on  the  first  of  two  LCCM  spreadsheets. 

These  prorated  cost  estimates  can  be  used  as  input  to  FOMOA  for 
specific  "what  if"  exercises  that  require  optimization  to 
address  issues  such  as  those  associated  with  horseblanket 
drills . 


(a)  Current  Method  for  Prorating  Production 
Costs.  Ideally,  variations  in  production  costs  would  be 
accompanied  by  the  concurrent  changes  in  the  P92  cost  cate¬ 
gories.  This  is  not  possible  in  the  limited  timeframe  for 
addressing  a  Value  Added  issue.  If  production  quantities  are 
increased  or  decreased,  the  CQM  is  used  to  determine  the  total 
production  costs  for  those  quantities.  LCCM  redistributes  total 
production  costs  by  the  percentage  of  total  production  costs  for 
each  P92  production  cost  element.  These  factors  are  calculated 
for  the  BCE/ACP  source  data  in  the  first  spreadsheet.  For 
example,  if  2.03  Engineering  Changes  accounted  for  3  percent  of 
total  production  costs  on  the  BCE/ACP,  the  second  spreadsheet 
containing  the  revised  production  quantities  would  estimate  the 
P92  cost  element  2.03  Engineering  Changes  to  be  3  percent  of  the 
new  total  production  costs. 

(b)  Methodology  for  Prorating  Fielding  and 
Sustainment  Costs.  Again,  based  on  the  BCE/ACP  source  data 
in  the  first  spreadsheet,  per  unit/system  P92  costs  are  calcu¬ 
lated  for  each  cost  element.  The  second  spreadsheet  reflects 
the  revised  costs  for  new  fielding  and  sustainment  quantities 
simply  by  multiplying  the  new  quantities  by  the  per  unit  costs 
calculated  from  the  BCE/ACP  source  data  in  the  first 
spreadsheet . 

(c)  Calculation  of  Military  Construction,  Army 
(MCA  and  Development  Costs).  MCA  and  development  costs  are 
not  changed  with  variations  in  production  quantities.  In  most 
applications,  this  is  appropriate,  since  these  costs  would  not 
vary  significantly  with  relatively  small  production  quantity 
changes.  In  instances  where  quantities  are  changed  signif- 


6-9 


CAA-SR-91-9 


icantly,  MCA  and  development  costs  would  have  to  be  reevaluated 
based  on  the  idiosyncrasies  associated  with  the  particular 
system’s  development  and  MCA  requirement. 

6-4.  IMPLEMENT  ANALYTICAL  MODELS  AND  TOOLS 

a.  Identify  Data  Requirements 

(1)  Life  Cycle  Cost  Data.  Life  cycle  cost  data  are 
obtained  from  BCE  or  Army  cost  positions  (ACP)  in  order  to 
provide  cost  quantity  data  for  inclusion  in  the  VAA  methodology. 
BCEs  are  generated  by  the  weapon  system  program  managers  (PMs). 
The  CEAC  performs  independent  cost  estimates  (ICEs)  to  validate 
the  BCE.  Differences  between  the  BCE  and  ICE  are  reconciled  in 
a  cost  analysis  brief  (CAB).  The  CAB  results  in  an  ACP  used  to 
support  the  POM.  ACP  or  BCE  data  are  used  in  VAA  depending  on 
which  is  the  most  current.  Updated  BCEs  should  be  available  on 
an  annual  basis.  BCE  updates  also  occur  when  the  approved 
quantities,  operating  tempo,  or  other  cost  determining  factors 
for  a  particular  weapon  system  have  changed  significantly. . 

(2)  Inflation  Indices.  Inflation  indices  guidance 
are  provided  annually  from  AMC  in  the  form  of  a  memorandum.  AMC 
guidance  provides  inflation  indices  for  each  Army  appropriation 
as  well  as  distinguishing  aircraft,  missiles,  weapons/tracktd 
combat  vehicles,  and  ammunition  procurement.  The  inflation 
indices  or  escalation  factors  are  needed  to  convert  constant 
(e.g.,  FY  90)  dollars  to  current  (i.e.,  inflated  or  escalated) 
dollars.  The  indices  are  also  used  in  the  conversion  of  current 
dollars  to  constant  dollars.  The  indices  are  important  because 
VAA  requires  consistency  among  the  types  of  costs  used  in  order 
to  make  comparisons  between  alternatives. 

(3)  Minimum/Maximum  Production  Rates.  Rates  of 
production  data  are  required  as  constraints  in  the  FOMOA 
(optimization  module)  Model  and  can  be  obtained  from  Assistant 
Secretary  of  the  Army  for  Research,  Development,  and  Acquisition 
(ASARDA)  or  the  PMs.  Systems  can  only  be  produced  between  the 
parameter  of  the  minimum  sustaining  rate  and  the  maximum 
production  rate.  The  minimum  sustaining  rate  (MSR)  is  defined 
as  a  level  of  production  which  will  keep  a  production  line  open 
while  maintaining  a  responsive  vendor  and  supplier  base  (i.e., 
warm  production  base).  The  assumption  inherent  in  the  MSR  is 
that  the  plant  must  operate  one  shift,  8  hours  per  shift,  5  days 
a  week.  Conversely,  the  maximum  production  rate  is  defined  as  a 
level  of  production  which  maximizes  the  capacity  of  the  existing 
tooling  or  facilities  without  requiring  an  additional  investment 
to  increase  the  production  capacity.  This  definition  assumes 
three  shifts  of  8  hours  per  shift,  7  days  a  week. 
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b.  Obtain  Modeling  Capability 

(1)  The  Cost  Quantity  Model  (CQM) 

(a)  The  CQM  provides  a  practical  tool  to  estimate 
average  unit  production  coats  of  a  given  weapons  system.  It 
utilizes  experience  curve  theory  to  make  these  estimates.  CQM 
allows  the  user  to  specify  unit  production  cost  as  either  a 
function  of  the  quantity  produced,  or  of  the  quantity  produced 
and  production  rate.  A  complete  description  of  experience  curve 
theory  and  the  use  of  the  cost/quantity/rate  method  is  provided 
in  Appendix  F. 

(b)  The  CQM  Model  performs  two  principal  functions. 
The  first  is  to  estimate  the  experience  curve  for  a  given 
weapons  system.  This  is  done  through  the  use  of  historical 
production  cost,  quantity,  and  rate  data.  Once  the  experience 
curve  is  estimated,  the  model  yields  other  needed  calculations. 
This  includes  the  system's  first  unit  cost  and  the  experience 
curve  percentages  (see  Appendix  F  for  a  description  of 
experience  curve  percentages). 

(c)  The  estimates  obtained  in  the  first  function  of 
the  CQM  are  inputs  to  the  second.  The  second  function  of  CQM 
allows  production  costs  to  be  computed,  given  the  quantities 
desired.  The  model  also  allows  for  constrained  budgets  and 
other  factors  that  might  shift  the  underlying  experience  curve 
(such  as  technological  change,  contract  competition,  etc.).  The 
model  produces  estimates  on  the  average  and  cumulative  average 
unit  production  costs,  in  both  constant  and  current  dollars. 

(2)  The  Life  Cycle  Cost  Model  (LCCM) 

(a)  LCCM  is  a  modified  version  of  the  Information 
System  Life  Cycle  Cost  Model  developed  by  CEAC.  Modifications 
and  enhancements  were  made  to  conform  to  weapon  system  life 
cycle  cost  data  and  to  provide  the  capability  to  redistribute 
cost  elements  concurrent  with  variations  in  procurement 
quantities.  All  formulas  and  macros  were  written  using  Quattro 
Pro.  All  computations  and  final  values  are  workable  and 
accurate  in  Lotus  1-2-3  spreadsheets  with  the  exception  of  the 
macros  that  realign  the  columns  and  provide  two  graph  options . 
Therefore,  it  is  recommended  that  Quattro  Pro  be  used  when 
entering  data  and  performing  sensitivity  analysis  on  the  cost 
categories  and  procurement  quantities. 

(b)  LCCM  comprises  two  spreadsheets.  The  first 
spreadsheet  contains  a  blank  template  for  the  data  to  be  entered 
based  on  the  general  structure  and  organization  of  the  P92  cost 
categories  developed  for  BCE  and  ACP.  P92  cost  categories  are 
the  cost  codes  established  by  the  Office  of  the  Comptroller  of 
the  Army  as  part  of  the  integration  of  weapon  system  costing, 
programming,  and  execution  of  management  systems.  The  second 
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spreadsheet  is  linked  to  the  first  and  recomputes  the  new  major 
category  quantity  and  cost  values  of  each  of  the  cells  by  unit 
cost  based  on  the  proportion  of  the  previous  entries  of  the 
BCE/ACP  data.  A  display  of  the  LCCM  spreadsheet  with  the  P92 
cost  codes  utilized  is  shown  in  Appendix  F,  Figure  F-1. 

(c)  A  summary  of  the  five  major  cost  categories  is 
computed  by  year  and  for  the  life  of  the  system.  Allowance  was 
made  for  data  13  years  beyond  the  current  year  set  by  the  user; 
however,  more  cells  for  additional  outyears  can  be  added  when 
necessary.  A  summation  of  the  following  appropriations  is  made 
for  research,  development,  testing,  and  evaluation  (RDTE), 
Military  Construction  Army  (MCA),  Operations  and  Maintenance 
(OMA),  Procurement  Army  (PA),  and  Military  Pay  and  Allowances 
(MPA) .  Costs  for  associated  items  of  equipment  of  each  system 
are  entered  on  the  first  spreadsheet  and  unit  costs  for  each 
ASIDE  calculated  to  apply  to  new  production  quantities  on  the 
second . 


(d)  The  model  is  time-phased  by  appropriation,  and 
the  life  cycle  costs  of  the  weapon  system  are  computed  in  both 
constant  dollars  (then  year  dollars)  and  current  dollars 
(dollars  of  the  current  fiscal  year).  Based  on  the  current  year 
that  the  user  enters,  the  prior  year  and  the  previous  years  are 
computed  as  is  the  budget  year,  the  subsequent  POM  years,  and 
the  outyears.  Inflation  from  the  US  Army  Material  Command  is 
used.l  The  inflation  factors  are  recorded  on  the  spreadsheet  in 
protected  mode  and  can  only  be  changed  by  using  the  disable 
protection  command  in  Quattro  Pro  or  Lotus  1-2-3. 

c.  Experimental  Design.  The  average  unit  cost  for  the 
projected  production  quantity  for  each  affected  year  is 
calculated  from  the  estimated  first  unit  cost,  experience  curve, 
and  production  rate  curve  considering  the  prior  quantity 
produced.  The  average  unit  cost  for  affected  years  is  then  used 
as  input  to  FOMOA,  which  computes  the  optimal  procurement 
quantities.  CQM  is  run  again  to  compute  the  average  unit  costs 
for  the  optimal  quantities  generated  by  FOMOA. 

d.  Use  Models  and  Tools 

(1)  Illustrative  Example 

(a)  This  paragraph  provides  an  illustrative  example 
of  the  application  of  CQM  and  the  potential  for  use  of  the  LCCM 
Model.  The  M1A2  Abreuns  main  battle  tank  from  the  armor/ 
antiarmor  illustrative  case  study  will  be  used  to  demonstrate 
these  two  models.  Table  6-1  displays  the  CQM  inputs  for  the 
M1A2  tank. 


^See  memorandum.  Headquarters.  U.S.  Army  Material  Command.  Subject;  Inflation 
Guidance.  Issued  28  December  1989. 
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Table  6-lA.  CQM  Input  -  M1A2  Tank  ($  FY  90  in  $M) 


Inputs 

Curve  factors 

Ist  Unit  Cost 

58.00 

A=58.00 

Experience  Curve  (B%) 

90.0% 

B=  -0.15200 

Production  Rate  Curve 
(C%) 

81.0% 

C=  -0.30401 

Previous  Quantity 

7,786 

(1+B)  =  0.84800 

Table  6-lB.  CQM  Input  by  Year  -  M1A2  Tank  ($  FY  90  in  $M) 


Production  FY 

1991 

1992 

1993 

1994 

1995 

1996 

1997 

Proi  Shift  +/-  (%) 

20.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

Desired  Quantity 

261 

516 

516 

516 

516 

420 

181 

Constant  $  Budget 

1,179.6 

1,754.4 

1,709.0 

1,330.0 

1,150.0 

635.0 

Inflation  Factors 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.20 

1.20 

1.20 

1.20 

1.20 

1.20 

1.20 

(1)  The  experience  curve  (B)  and  the  production  rate 
curve  (C)  were  estimated  from  historical  production  data  for  the 
Ml  tank  series  using  the  cost/quantity/rate  method  described  in 
detail  in  Appendix  F,  paragraph  F-5.  The  first  unit  cost  was 
estimated  to  be  S58M,  the  cost/quantity  curve  at  90  percent,  and 
the  production  rate  curve  at  81  percent.  Prior  quantities  of  Ml 
tanks  produced  were  7,786.  The  desired  annual  production 
quantities,  with  the  projected  annual  budget  dollars  are  input 
into  the  model,  and  the  average  annual  unit  production  cost  (in 
FY  90  constant  dollars  for  this  example)  is  computed  (see  Table 
6-3).  These  average  annu  L  unit  production  costs  are  then  used 
as  input  to  the  FOMOA  Modv.  1  which  computes  the  optimal  produc¬ 
tion  quantities  considering  both  budget  dollars  available, 
combat  effectiveness,  and  competing  alternative  weapon  systems. 
Once  the  optimal  M1A2  quantities  were  computed  by  FOMOA,  the 
average  unit  production  costs  for  the  optimized  production 
quantities  were  calculated  by  a  second  application  of  the  CQM. 

A  20  percent  projected  shift  factor  was  included  in  the  first 
year  of  production  of  the  M1A2  (1991)  to  account  for  the  average 
unit  production  cost  delta  for  the  M1A2  tank  over  the  MlAl  tank. 

(2)  The  LCCM  input  is  extracted  from  BCE/ACP  data  and 
entered  into  the  Life  Cycle  Cost  Model.  The  input  format  for 
the  LCCM  is  shown  in  Tables  6-2A  through  6-2E. 
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Table  6-2A.  LCCM  Input  Data  -  Development  Costs  ($M) 


Category  I  Cost  element 


Facilities 


Other  Dev  Costs 


Table  6-2B.  LCCM  Input  Data  -  Production  Costs  ($M) 
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Table  6-2C.  LCCM  Input  Data  -  MCA  and  Fielding  Costs  ($M) 


Category 

Cost  element 

Appn 

H 

Prior 

year 

1989 

Current 

year 

1990 

Budget 

year 

1991 

POM 
yr  1 
1992 

POM 
yr  2 
1993 

Etc. 

3.0 

MCA 

X 

X 

X 

X 

X 

X 

X 

4.0 

Fielding 

OMA 

X 

X 

X 

X 

X 

X 

X 

4.01 

System  Test 

OMA 

X 

IHSi 

X 

X 

X 

X 

X 

4.02 

Training, 

Serv/Equipment 

OMA 

X 

■■ 

X 

X 

X 

m 

X 

4.03 

OMA 

X 

X 

X 

X 

X 

X 

X 

4.04 

Intial  Repair  Part 

OMA 

X 

X 

X 

X 

X 

X 

X 

4.041 

Intial  Repair  Part 

OMA 

X 

X 

X 

X 

X 

X 

X. 

4.0411 

OMA  Funded 

OMA 

X 

X 

X 

X 

X 

X 

X 

4.0412 

Procurement  Funded 

PA 

X 

X 

X 

X 

X 

X 

X 

4.042 

OMA 

X 

X 

X 

X 

X 

X 

X 

4.05 

Sys  Spec  Base 

OMA 

X 

X 

X 

X 

X 

X 

X 

4.051 

Civilian 

OMA 

X 

X 

X 

X 

X 

X 

X 

4.052 

Military 

OMA 

X 

X 

X 

X 

X 

X 

X 

4.053 

Contractor 

OMA 

X 

X 

X 

X 

X 

X 

X 

4.054 

Other 

OMA 

X 

X 

X 

X 

X 

X 

X 

4.06 

Other  Fund  Field 

OMA 

X 

X 

X 

X 

X 

X 

X 

4.061 

Other  OMA  Fund 

OMA 

X 

X 

X 

X 

X 

X 

X 

4.062 

Other  PA  Fund 

PA 

X 

X 

X 

X 

X 

X 

X 
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Table  6-20. 


LCCM  Input  Data  -  Sustainment  Costs  ($M) 


Category  I  Cost  element 


5.011 


5.0111 


5.0112 


5.012 


5.0121 


5.0122 


5.014 


5.0141 


5.0142 


5.015 


5.0151 


5.0152 


5.03 


5.031 


5.032 


5.0411 


5.0412 


5.0412 


5.0412 


5.0412 


Sustainment 


Replenishment 


Replenish  Repair 


OMA  Funded 


Procurement  Funded 


OMA  Funded 


Procurement  Funded 


War  Reserve  Repair 


War  Reserve  Spares 


OMA  Funded 


Procurement  Funded 


lESSBSQBSaa 


OMA  Funded 


Procurement  Funded 


Ammo/Missile 


Train-Ammo  /  Msl 


War  Reserve 
Ammo /Msl 


Depot  Maint 


Depot  Maint  Civil 


veh  OH  Civl  Lab 


veh  OH  Mat  Lab 


Civilian 


5.072 


Ammo /Msl /Equipment 


System/Trnq/Srv 
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Table  6-20.  (continued)  LCCM  Input  Data  -  Sustainment 

Costs  ($M) 


Category 

Cost  element 

Appn 

Total 

previous 

years 

Prior 

year 

1989 

Current 

year 

1990 

Budget 

year 

1991 

POM 
yr  1 
1992 

POM 
yr  2 
1993 

Etc  . 

5.08 

MPA 

MPA 

X 

X 

X 

X 

X 

X 

X 

5.09 

Sys/Proj  Mqmt  Civ 

PA 

X 

X 

X 

X 

X 

X 

X 

5.10 

Modification  Kits 

PA 

X 

X 

X 

X 

X 

X 

X 

5.101 

Hardware 

PA 

X 

X 

X 

X 

X 

X 

X 

5.102 

Application 

PA 

X 

X 

X 

X 

X 

X 

X 

5.111 

Other  OMA  fund 

OMA 

X 

X 

X 

X 

X 

X 

X 

5.112 

PA 

X 

X 

X 

X 

X 

X 

X 

Table  6-2E.  Link  Between  Appropriation  Summary  and  Major 

Cost  Categories 


Category 

Appn 

Hi 

Prior 

year 

1989 

Current 

year 

1990 

Budget 

year 

1991 

POM 
yr  1 
1992 

POM 
yr  2 
1993 

Etc. 

Development 

RDTE 

X 

X 

X 

X 

X 

X 

X 

Production 

OMA 

PA 

X 

m 

X 

X 

X 

m 

m 

MCA 

X 

X 

X 

X 

X 

X 

Fielding 

OMA 

PA 

X 

X 

X 

X 

X 

X 

■ 

Sustainment 

OMA 

MPA 

PA 

X 

■ 

X 

■ 

X 

■ 

X 

NOTES: Constant  or  Current  SM 

X 

X 

X 

X 

X 

X 

6-5.  ANALYZE  RESULTS 

a.  Introduction.  This  paragraph  provides  the  CQM  inputs 
and  outputs  for  the  M1A2  tank  illustrative  example.  Also 
included  is  the  incorporation  of  an  application  of  the  LCCM  and 
the  results  of  an  ARIM  type  weapon  system  definition  (using  the 
M1A2  tank  illustrative  example). 

b.  CQM  Results.  Table  6-3  provides  the  CQM  results  for 
the  planned  procurement  quantities  presented  in  the  May  1989 
M1A2  ACP.  Average  unit  costs  declined  until  the  last  2  years 
when  the  reduced  quantities  increased  the  average  unit  costs 
(Table  6-4),  The  CQM  estimates  were  within  13  percent  of  the 
costs  generated  by  the  ACP. 
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Table  6-3.  CQM  Output  M1A2  -$  FY90/$M  (A3  example) 


XEAR 

1991 

1992 

1993 

1994 

1995 

1996 

1  9  9  7  li 

PROPOSED  RATE 

261 

516 

516 

516 

516 

420 

HBB 

ACTUAL  1ST  ITEM 

1.00 

262.00 

778.00 

1294.00 

1810.00 

2326.00 

2746.00 

ACTUAL  LAST  ITEM 

261.00 

777.00 

1293.00 

1809.00 

2325.00 

2745.00 

2926.00 

ACTUAL  PROD  QTY 

261.00 

516.00 

516.00 

516.00 

516.00 

420.00 

161.00 

CUMUL  ACTUAL  QTY 

261.00 

777.00 

1293.00 

1809.00 

2325.00 

2745.00 

2926.00 

ACT  PROD  CONSTANT  S 

855.00 

1364.10 

1353.70 

1340.00 

1329.10 

1143.60 

633.80 

CUMUL  CONSTANT  S 

855.00 

2219.20 

3570.80 

4910.90 

6240.00 

7383.60 

8017.40 

ACTUAL  PROD  CURR  $ 

855.00 

1364.10 

1353.70 

1340.00 

1329.10 

1143.60 

633.80 

CONSTANT  $  AVG  UNIT 

PROD  COST 

3.28 

2.64 

2.62 

2.60 

2.58 

IHfl 

3.50 

Table  6-4.  Average  Unit  Productio.i  Costs  of  Optimized 

Qtys  -  M1A2 


YEAR 

1995 

1996 

1997 

1998 

1999 

2  000 

OPTIMIZED  QTYS 

64 

64 

64 

64 

54 

AVG  UNIT  PROD  COSTS 

S5.03 

S5.03 

$5.02 

$5.01 

$5.01 

$5.00| 

The  CQM  provided  a  useful  tool  for  the  quick  reaction  cost 
quantity  data  required  by  the  Value  Added  Analysis  methodology. 
The  Cost  Quantity  Model  has  been  used  successfully  in  conjunc¬ 
tion  with  FOMOA  to  optimize  acquisition  strategies  for  several 
modernization  plans,  including  armor  systems  modernization  (ASM) 
and  helicopters.  FOMOA  cannot  currently  dynamically  account  for 
changes  in  unit  weapon  system  costs  as  a  result  of  experience 
and  production  rate  curves.  CQM  accounts  for  this  by  providing 
the  adjusted  unit  production  costs  by  year  for  the  optimal  quan¬ 
tities  computed  in  an  initial  FOMOA  run  (based  on  an  average 
unit  cost  FOMOA  is  then  rerun  utilizing  the  adjusted  unit  costs, 
considering  the  experience  and  production  curves.  The  result  is 
a  more  accurate  optimization  of  procurement  quantities,  by  year, 
given  constrained  annual  budgets.  The  feasibility  of  incorpo¬ 
ration  of  the  CQM  or  CQM  methodology  into  FOMOA  directly  may  be 
addressed  in  the  future. 

c.  The  LCCM  output  provides  the  same  cost  elements  as  the 
input  shown  in  Tables  6-2A  through  6-2E.  The  cost  elements  that 
relate  to  the  five  major  weapon  system  type  definitions  for  the 
M1A2  tank  are  shown  in  Table  6-5 
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Table  6-5.  Major  Weapon  System  Cost  Definitions  -  M1A2 


1  MAJOR  WEAPON  SYSTEM  DEFINITION 

LCCM  COST  ELEMENTS 

Ply  Away  Cost 

2.0213  warranty  +  2.0214  other 

Engineering 

2-022  rec  engineering 

Tooling 

2-023  tooling 

Quality  Control 

2-024  quality  control 

Nonrecurring  "Star-  Up" 

2.01  nonrecur~ing 

Allowance  for  Changes 

2.03  engineering  changes 

2. 05  system  test  &  evaluation 

2.094  other  production  cost 

Weapon  Syatan  Costs 

Weapon  System  Costs 

Fly  Away  Costs  Plus 

Technical  Ccta 

2.04  data 

Contractor  Service 

2.0211  contract 

Support  Equipment 

2.09  critical  ASIDE 

Training  Equipment/Fc. .  tory  Training 

2.08  operational  site  activity 

Procurement  Costs 

Procurement  Costs 

Weapon  System  Costs  Plus 

Weapon  System  Costa  Plus 

Initial  Spares 

2.07  initial  spares 

Other 

4.0  fielding  (procurement) 

Program  Acquisition  Coats 

Program  Acquisition  Costs 

Procurement  Costa  Plus 

Procurement  Costs  Plus 

P.JT4E 

Facility  Construction 

1.0  development 

3.0  military  construction 

Life  Cycle  Costs 

Life  Cycle  Costs 

Program  Acquisition  Coats  Plus 

program  Acquisition  Costs  Plus 

Operations  &  Support 

5.0  sustainment 

common  Support  Equipment 

Disposal 

Total  life  cycle  coats 

6-6.  SUMMARY  AND  OBSERVATIONS 

a.  CQM  and  LCCM  are  models  for  use  in  the  Value  Added 
methodology  for  estimating  weapon  system  costs,  given  variable 
budget  constraints  ana  system  requirements.  The  two  cost  models 
developed  are  capable  of  estimating  impacts  on  cost  categories 
and  appropriations  for  a  large  number  of  weapon  systems  in  a 
short  timeframe.  The  ARIM  Study  clarified  the  five  major 
definitions  of  weapons  systems  which  will  be  incorporated  in  the 
Value  Added  Costing  methodology. 

b.  The  LCCM  combined  with  the  CQM  provides  a  useful  tool  for 
estimating  the  impact  of  variations  in  production  quantities  on 
the  various  life  cycle  cost  elementj  and  affected  appropri¬ 
ations.  Additional  sensitivity  analysis  using  the  LCCM  and  CQM 
should  be  conducted.  In  addition,  the  LCCM  capabilities  should 
be  expanded  to  provide  the  five  major  weapon  system  cost 
definitions  shown  in  Table  6-5. 
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c.  The  CQM  and  LCCM  provide  useful  tools  for  the  quick 
reaction  cost  analysis  required  in  the  budget  process.  CQM  is  a 
mechanism  for  the  estimation  of  cost/quantity  and  cost/rate 
relationships.  LCCM  approximates  the  impact  of  variations  in 
procurement  quantities  on  the  various  subcategories  of  cost 
elements  and  affected  appropriations.  Both  of  these  models 
provide  cost  data  that  can  be  incorporated  into  the  optimization 
model  if  required. 

d.  There  is  an  absence  of  a  central  data  base  containing 
weapon  system  cost  data  at  either  the  BCE  level  of  detail  or  at 
the  summary  level  of  detail.  This  data  is  required  for  Value 
Added  applications  as  well  as  numerous  other  applications 
throughout  the  Army.  The  responsibility  for  collecting  and 
accessing  data  is  consistent  with  the  missions  and  functions  for 
CEAC.  Currently,  this  data  is  forwarded  to  or  generated  by 
CEAC,  but  there  is  no  mechanism  in  place  for  centralizing  weapon 
system  data.  A  recent  effort  by  AMC  to  install  standardized 
automated  BCE  formats  at  the  program  managers  (PMs)  should 
provide  impetus  for  the  centralization  of  BCE  data. 
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^CHAPTER  7 

IMPLICIT  MEASURES  OF  EFFECTIVENESS 


7-1.  INTRODUCTION 

a.  The  purpose  of  this  chapter  is  to  provide  a  discussion  of  the 
formal  analytical  framework  used  to  perform  quantitative  analysis  of 
subjective  data  which  we  call  implicit  measures  of  effectiveness. 

This  chapter  also  provides  a  detailed  discussion  of  how  these 
implicit  values  will  be  used  in  the  Value  Added  Analysis.  This 
chapter  is  structured  as  follows: 


(1)  Introduction 

(2)  SIAM  Factors 

(3)  Summary 

b.  Figure  7-1  depicts  the  Implicit  Effectiveness  Module  and 
how  it  relates  to  the  overall  VAA  methodology.  The  overview 
chart  indicates  that  there  are  two  elements  within  the  module,  a 
leadership  survey  element  and  a  decision  element.  The 
leadership  survey  is  conducted  to  obtain  weights  for  use  in  the 
effectiveness  integration  module  and  in  helping  define  the 
secondary  factors.  The  secondary  factors  decision  box  is 
present  within  the  module  to  indicate  that  the  VAA  methodology 
at  this  point  requires  a  decision.  The  analyst  may,  at  this 
point  in  the  analysis,  decide  to  include  or  exclude  the  SIAM 
factors  as  part  of  the  single  measure  of  value. 
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Figure  7-1.  VAA  Methodology  -  Implicit  Effectiveness 

Overview 

c.  The  study  team  decided  early  in  the  Value  Added  Analysis 
to  quantify  subjective  judgments  because  of  the  following^: 

(1)  A  formal  analytical  framework  supports  decision¬ 
makers  in  their  attempt  to  consider  relevant  information 
systematically  and  to  examine  options  and  consequences  one  at  a 
time. 


(2)  Analysts  can  break  an  unmanageable  problem  into 
manageable  parts  and  then  synthesize  information  about  the  parts 
in  a  rational  fashion. 

(3)  A  formal  framework  allows  sensitivity  analysis  on 
alternative  judgments,  tradeoffs  among  alternative  judgments, 
study  of  effects  of  variations  in  subjective  judgments  on 
outcomes;  and  it  is  repeatable. 

This  group  of  implicit  measures  (subjective  judgments)  are 
collectively  called  SIAM  factors. 

7-2.  SIAM  FACTORS 

a.  Development  of  the  Context  for  SIAM 

(1)  As  indicated  in  paragraph  2-2,  we  defined  value  as 
being  contextual.  We  have  already  discussed  in  Chapter  5  some 
of  the  most  direct  contextual  items.  These  included  the 


^Crawford,  Gordon  and  Cindy  Williams.  The  Analysis  of  Subjective 
Judgment  Matrices.  RAND  Document  R-2572-1-AF,  May  1985  (AD-A157 
399)  (UNCLASSIFIED),  page  2. 
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identification  of  theaters  of  interest,  doctrine,  tactics, 
weapons  systems,  and  model  assumptions.  All  of  these  items  were 
used  to  obtain  the  explicit  measures  of  effectiveness  through 
combat  modeling.  However,  in  analyzing  how  program  decisions 
are  actually  made  within  the  Pentagon  and  the  Department  of  the 
Army,  it  became  very  clear  that  other  factors  also  influenced 
the  decisions.  These  factors  became  known  as  SIAM  factors. 

(2)  The  SIAM  factors  do  reflect  context.  The  manner  in 
which  context  is  achieved  is  twofold.  First,  context  is  created 
as  a  function  of  who  decides  what  is  a  SIAM  factor,  i.e.,  which 
special  interest  group,  such  as  the  Congress,  Office  of 
Management  and  Budget  (0MB),  Office  of  the  Secretary  of  Defense 
(OSD),  CINCs,  ARSTAF,  or  major  Army  command  (MACOM)  commanders. 
Second,  these  factors  provide  context  through  groups  weighting 
each  of  the  factors  so  that  they  reflect  the  group's  value 
judgments.  Context  in  this  sense  refers  to  what  foreign  and 
domestic  issues  impact  upon  the  decisionmakers  and  how  they 
assimilate  these  issues  to  make  judgments. 

b.  Terms  and  Definitions.  This  paragraph  is  designed 
to  provide  the  reader  an  understanding  of  the  lexicon  used  in 
developing  the  implicit  measures  of  effectiveness. 

(1)  Combat  Effectiveness.  The  contribution  to 
war fighting  capability. 

(2)  Noncombat  Effectiveness.  The  contribution  to 
warfighting  capabilities  other  than  warfighting.  This  term 
includes  such  factors  as  nation  building,  operating  facilities 
to  support  soldier  quality  of  life  and  contributions  to  special 
programs  of  national  interest  such  as  fighting  drugs,  en¬ 
vironmental  protection  and  energy  conservation. 

(3)  Secondary  Impact  Analysis  Modifiers.  Key 

factors  which  are  considered  in  the  decisionmaking  process,  but 
are  not  official  Army  measures  of  effectiveness.  For  example, 
political  risk. 

(4)  Blue  Force  Surviving.  A  measure  of  combat 
effectiveness  obtained  from  a  combat  simulation.  It  is  the 
percentage  of  the  Blue  combat  power  remaining  after  the  end  of  a 
Red  first  echelon  attack. 

(5)  Correlation  of  Forces  and  Means.  A  measure  of 
combat  effectiveness  obtained  from  a  combat  simulation.  This 
measure  is  computed  as  the  ratio  of  Red  and  Blue  combat  power  in 
the  strike  sector  remaining  after  the  end  of  a  Red  first  echelon 
attack.  It  is  commonly  used  to  infer  the  denial  of  the  enemy's 
tactical  objectives. 

(6)  Movement  of  Force  Center  of  Mass.  A  measure  of 
combat  effectiveness  obtained  from  a  combat  simulation.  When 
Blue  is  on  the  defensive,  it  is  a  geographical  measure  of  loss 
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of  territory.  Minimum  movement  is  commonly  used  to  infer  the  - 
accomplishment  of  a  Blue  defensive  objective.  When  Blue  is  on 
the  offensive,  success  is  measured  as  movement  toward  the 
objective. 

(7)  Contribution  to  industrial  Preparedness.  A 
measure  of  the  impact  of  a  proposed  program  on  the  ability  of 
the  industrial  base  of  the  nation  to  respond  to  extraordinary 
production  requirements . 

(8)  Political  Risk:  Public  Opinion.  The  public 

opinion  measure  of  political  risk  is  a  subjective  evaluation  of 
the  attitudes  of  the  general  public  toward  a  proposed  program. 
It  ranges  from  significant  positive  support  for  the  program  to 
significant  opposition  to  the  program. 

(9)  Political  Risk:  Congressional  Opinion.  The 

Congressional  opinion  measure  of  political  risk  is  a  subjective 
evaluation  of  the  attitudes  of  the  Congress  toward  a  proposed 
program.  It  ranges  from  significant  positive  support  for  the 
program  to  significant  opposition  to  the  program. 

(10)  Political  Risk:  Executive  Branch  Opinion. 

The  executive  branch  opinion  measure  of  political  risk  is  a 
subjective  evaluation  of  the  attitudes  of  the  0MB,  OSD,  and  JCS 
toward  a  proposed  program.  It  ranges  from  significant  positive 
support  for  the  program  to  significant  opposition  to  the 
program. 

(11)  Political  Risk:  Internal  Army  Opinion.  The 

internal  Army  opinion  measure  of  political  risk  is  a  subjective 
evaluation  of  the  attitudes  of  special  interest  groups  within 
the  Army  toward  a  proposed  program.  It  ranges  from  significant 
positive  support  for  the  program  to  significant  opposition  to 
the  program. 

(12)  Institutional  Stability.  A  measure  of  the 
amount  of  change  which  is  associated  with  a  program.  Change  is 
measured  in  terms  of  the  impact  on  the  personnel,  logistical, 
and  training  systems  of  the  Army.  Change  is  subjectively 
measured  on  a  scale  ranging  from  extreme  change  to  marginal 
change . 


(13)  Program  Flexibility.  A  measure  of  the  funding 
flexibility  associated  with  a  program.  In  other  words,  it 
measures  how  much  latitude  is  available  to  reprogram  resources 
or  change  decisions  with  regard  to  the  program.  For  example,  a 
multiyear  contract  with  a  significant  penalty  clause  would  have 
low  flexibility. 

(14)  Program  Feasibility.  A  subjective  measure  of 
the  number  of  obstacles  which  might  prevent  progreim  execution. 
Programs  which  are  extremely  complex  or  slow  to  implement  have 
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negative  ratings  for  feasibility,  while  programs  with  few 
bureaucratic  hurdles  have  positive  ratings  for  feasibility. 

(15)  Asset  Versatility  and  Deployability.  Asset 
versatility  and  deployability  measures  the  applicability  of  a 
program  to  multiple  theaters  of  operation.  For  example,  a 
program  that  bought  new  weapons  that  could  be  used  in  all 
theaters  wou^d  have  relatively  high  ratings  for  asset 
versatility. 

(16)  Operational  and  Technical  Risk.  Operational 
and  technical  risk  is  a  subjective  measure  of  the  probability 
associated  with  a  program  meeting  all  of  its  stated  performance 
criteria.  For  materiel  systems,  this  factor  will  roughly 
correspond  to  the  stage  of  materiel  development  for  the  system. 
For  example,  a  system  in  6.1  will  have  a  higher  risk  than  a 
system  in  stage  6.3. 
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(17)  Need  as  Related  to  Current  Capability. 

Program  need  is  a  subjective  measure  of  the  current  status  of 
the  mission  area  or  function  associated  with  a  program,  and  the 
relative  fraction  of  improvement  that  is  expected.  For  example, 
programs  that  only  enhance  an  adequate  capability  receive  a  more 
negative  rating  for  need  than  those  which  address  an  area  that 
has  no  current  capability. 

c.  Decisionmakers  (DMs)  and  the  Pairwise  Comparison 
Approach.  Experts  in  decision  theory  recognize  the  Pairwise 
comparison  approach,  used  in  the  Analytic  Hierarchy  Process 
(AHP)  and  developed  by  Thomas  L.  Saaty  in  the  1970s,  as  being 
particularly  effective  in  eliciting  experts'  subjective 
judgments  free  from  the  usual  negative  effects  of  group 
pressure.  As  discussed  in  Chapter  1,  some  facets  of  the  problem 
of  developing  the  most  desired  POM  lack  any  well-defined, 
scalar-valued  measures  of  merit  and  involves  a  great  many 
interrelated  issues  such  as:  combat  MOEs,  noncombat  MOEs,  and 
SIAM  MOEs. 

(1)  An  initial  decision  hierarchy  (strawman),  which 
allows  the  DM  to  break  the  complex  problem  into  its  constituent 
elements,  is  developed  using  literature  reviews,  brainstorming, 
and  expert  opinion. 

(2)  A  final  decision  hierarchy  is  constructed  with 
validated  factors  only  after  considerable  discussion  of  the 
strawman  with  functional  experts,  the  sponsor  and  various  other 
DMs.  The  overall  objective  is  the  only  element  in  Level  One. 
Level  Two  consists  of  the  judgment  criteria  the  DMs  consider 
important  to  the  decision;  each  criteria  can  be  further 
subdivided.  Level  Three  contains  the  attributes  (elements)  of 
the  previous  higher  level. 
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Figure  7-2 


VAA  Analytical  Hierarchy 


(3)  The  solicitation  of  judgments  will  be  accomplished 
by  means  of  a  deskside  briefing  on  the  Value  Added  Analysis 
Study,  followed  by  a  survey  of  the  attributes.  The  survey 
participants  should  cover  a  broad  range  of  Department  of  the 
Army  specialties,  all  required  to  make  daily  decisions  relating 
to  the  Army  program.  During  this  phase  of  the  study,  the  DM 
makes  his  or  her  preferences  known,  using  a  modified  version  of 
Saaty's  AHP  comparison  scale  (Figure  7-2).  All  participants 
should  receive  the  sautie  survey.  However,  there  is  no  necessity 
for  comparisons  to  be  in  the  same  sequence  and  in  fact  should  be 
random  if  possible  as  a  means  of  dealing  with  bias. 
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\f/y'  ♦ 

I  !  Equal  Importance  of  Both  Elements  X  1 

I  /  Weak  Importance  of  One  Element  over  Anothe^^I^^^ 


(t  ■  Strong  Importance  of  One  Element  over  Anothdl  "  X  1 

Very  Stong  Importance  of  One  Element  Over  Another  ^  1 

Absolute  Importance  of  One  Element  Over  Anothui.'  '  ■  * 

NoteO  Indicates  Intermediate  Values 


Figure  7-3.  SAATY'S  Pairwise  Comparison  Scale 

(4)  The  establishment  of  priorities  is  accomplished  by 
making  pairwise  comparisons,  that  is  "to  compare  the  elements  in 
pairs  against  a  given  criterion."^ 

(5)  A  pairwise  comparison  matrix  is  developed  for  each 
respondent  after  conducting  the  confidential  interviews  to 
determine  the  order  of  attribute  priority  with  relative  weights . 
The  study  team  used  a  set  of  survey  instruments  such  as  the  one 
depicted  in  Figure  7-3. 


SURVEY  INSTRUMENT 


Contribution  to 
Industrial  Preparedness 

The  contribution  to  industrial  preparedness 
measures  the  impact  of  a  proposed  program 
on  the  ability  of  the  industrial  base  of  the 
nation  to  respond  to  extraordinary  produc¬ 
tion  requirements. 


Political  Risk: 

Public  Opinion 

The  public  opinion  measure  of  political 
risk  is  a  subjective  evaluation  of  the  atti¬ 
tudes  of  the  general  public  toward  a  pro¬ 
posed  program  It  ranges  from  significant 
positive  support  for  the  program  to  signifi¬ 
cant  opposition  to  the  program. 


□  □□□nnnnDnnnnnnnn 

Check  the  box  which  you  believe  is  presently  the  best  measure  of  the 
importance  of  the  item  on  the  left  as  compared  to  the  item  on  the  right. 

Figure  7-4.  Example  Survey  Instrument 


1  Saaty,  T.  L.  Decision  M 
Decisions  in  a  Complex  \ 
(UNCLASSIFIED)  Page  76. 


tic  Hiera 


Belmont,  CA:  LifetimeLearning  Publications,  1982 
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(6)  The  following  is  an  illustrative  example  of  a  pair¬ 
wise  comparison  and  the  calculation  of  the  weights.  Figure  7-4 
is  an  example  of  a  tradeoff  issue  in  which  the  decisionmaker  is 
seeking  the  most  cost  effective  mix  of  armor  and  antiarmor  (A3) 
systems . 


V 


Example:  Representative  Armor/ Anti-Armor  Systems 


Issue:  Given  limited  dollar  resources,  which  mix  of  A3  Systems 
should  be  procured? 

Alternatives  Characteristics 

Basecase  Europe  Scenario  6.3  with  25  Systems 

Alt  1  •  M1A2 

•  NLOS-AT 

•  MLRSSADARM 

Alt  2  •  M1A2 

■  AMSH 

•  BRADLEY  BUCm 
’  MLRSSADARM 

_ J 


Figure  7-5.  Example  Tradeoff  Issue 


Table  7-1  is  how  one  would  form  a  matrix  for  comparing  the 
issues  as  outlined  in  the  previous  figure.  The  numeric  values- 
show  the  relative  importance  to  the  decisionmaker  of  the  three 
sample  SIAM  factors  in  relation  to  the  mix  of  A3  systems. 


Table  7-1.  Example  Decision  Matrix 


Political  Risk 

Congressional 

Opinion 

Asset  Versa¬ 
tility  and 
Deployability 

Operational  and 
Technical  Risk 

Political  Risk 

Congressional 

Opinion 

1 

1/2 

1/5 

Asset  Versa¬ 
tility  and 
Deplovabilitv 

2 

1 

1/2 

Operational 
and  Technical 
Risk 

5 

2 

1 

(7)  Priority  weights  are  computed  by  the  geometric  mean 
method,  rather  than  the  more  commonly  used  eigenvector  method, 
because  it  gives  rise  to  a  more  meaningful  measure  of  consis¬ 
tency  with  known  statistical  properties.  The  RAND  study^ 


1  Crawford,  Gordon  and  Cindy  Williams.  The  Analysis  of  Subjective 
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appropriately  points  out  that  the  geometric  mean  vector  is 
"rooted  in  a  mathematical  approach  to  estimation,"  providing  an 
intuitive  understanding  to  the  problem  as  well  as  means  to 
assess  the  method's  suitability.  The  geometric  mean  is  computed 
as  follows: 


STEP  1:  Compute 


V 


fori  -  1,2, 


n 


[7.01] 


where : 

Vi  -  Non  -  normalized  weight  for  attribute 

n  =  total  number  of  attributes, 

i  =  row  attribute  index, 

j  =  column  attribute  index, 

aii  =  value  for  position  ij  in  decision  matrix  A, 

V  =  vector  of  geometric  means. 


STEP  2:  For  1/ such  that 


[7.02] 


[7.03] 


■((1)  (1/2)  (1.6))'^^' 

0.4641589' 

((2)  (lUl/2))’^^ 

= 

1.0000000 

((5)  (2)  (1))’^^ 

2.1544347 

STEP  3;  Compute  the  sum  of  the  elements  of  V, 


3.6185936 


[7.04] 


Judgment  Matrices.  RAND  Document  R-2572-1-AF,  May  1985  (AD-A157 
399)  (UNCLASSIFIFD).  Page  6, 
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(8)  The  normalized  geometric  mean  vector,  Vn'  is 
computed  by  dividing  each  element  of  the  geometric  mean  vector 
by  the  sum  of  all  the  elements  in  the  geometric  mean  vector. 

The  sum  of  the  elements  of  the  normalized  geometric  mean  vector 


is  one. 

STEP  4 ;  Compute  Vn  as 

■JL 

V 

ISK 


•0.4641589' 

3.6185936 

1.0000000 

3.6185936 

2.1544347 

3.6185936 


[7.05] 


■v,„- 

V3„ 


[7.06] 

0.1282705' 

0.2763505 

0.595379 


STEP  5:  Check  the  sum  of  the  elements  of  Vn, 


[7.07] 


-  0.1282705  +  0.2763505  +  0.595379  -1 


(9)  Though  consistency  is  not  required  from  the  DM  when 
making  pairwise  comparisons,  we  must  concern  ourselves  with 
knowing  the  degree  of  his  consistency: 

"...Consistency  ...informs  the  judges  about  the 
adequacy  of  their  knowledge  and  whether  they  need 
to  study  the  matter  further  in  order  to  obtain 
greater  coherence  in  their  understanding  of  the 
problem . ^ " 


'Wind,  Y.  and  T.  L.  Saaty.  "Marketing  Applications  of  the 
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(a)  We  do  not  want  the  decision  to  be  based  on 
judgment  with  low  consistency  with  results  of  the  survey 
appearing  to  be  random.^  Thus,  a  certain  degree  of  consistency 
is  needed  to  obtain  valid  results.  Low  consistency  or  high 
inconsistency  may  also  indicate  that:  (i)  the  decisionmaker 
dees  not  understand  the  pariwise  comparison  process,  (2)  the 
decisionmaker  did  not  take  necessary  time  to  perform  the 
comparisons,  or  (3)  the  decisionmaker  has  internal  conflicts  or 
inconsistencies  himself.  A  consistency  ratio  (CR)  measures  the 
overall  consistency  of  judgments  and  provides  an  indication  that 
the  decisionmaker's  values  did  not  change  dramatically  during 
the  rating  process.  It  is  through  the  proportionality  of  the 
preferences  that  the  decisionmaker  indirectly  provides  this 
measurement. 

(b)  The  CR  is  calculated  as  the  consistency  index  (Ci) 
divided  by  the  average  random  index  (RI).  It  is  expressed  as: 

[7.08] 


A  CR  of  0.10  or  less  is  considered  acceptable;  a  CR  greater  than 
0.10  makes  the  judgments  appear  random  and  indicates  a  need  to 
improve  the  judgments  or  to  restructure  the  hierarchy. ^  when 
the  CR  is  zero,  you  have  complete  consistency.  The  goal  is  not 
to  minimize  the  CR,  but  to  make  good  sound  judgments  and 
decisions.  The  principle  or  maximum  eigenvalue,  Imax' 
number  of  elements  for  a  given  matrix  determine  the  consistency 
index  (closeness  to  consistency),  or  inconsistency  index  as  it 
is  sometimes  called.  This  consistency  index  is  represented  by 
the  following  expression:'' 


[7.09] 


^max  ^ 

n-1 


where, 


A 


the  maximum  eigenvector  (sum  of  the  product  of 
the  pairwise  comparison  matrix  and  the  normalized 


Analytic  Hierarchy  Process. "Management  Science.  26:  641-658 
(July  1980),  Page  647. 

^Saaty,  T.  L.  The  Analytic  Hierarchy  Process.  New  York:  McGraw- 
Hill  Book  Company,  1980,  page  21. 

^ibid..  Page  21. 

^ibid..  Page  21. 

“^ibid..  Page  21. 
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geometric  mean  vector). 

n  =  the  order  of  the  pairwise  comparison  inatrix  (the 
number  of  activities). 


(c)  Let  A  equal  the  pairwise  comparison  matrix  before 

weighting.  To  determine  the  value  of  in  the  illustrative 

example,  the  pairwise  comparison  matrix  A  (3x3)  is  multiplied  by 
the  normalized  geometric  mean  vector  (3x1)  and  the  elements  of 
that  vector  are  summed  as  shown,  resulting  in  a  scaler  value 

^  mzx  , 


STEP  1;  Compute 


[7.10j 


AV^n-'^nA- 


VinA 

VanA 

VanA 


giving 


[7.11] 


■1 

1/2 

1/51 

[0.1282705  ■ 

[0.3855216] 

2 

1 

1/2 

X 

0.2763505 

= 

0.830581 

5 

2 

1 

0.595379 

1.789433 

STEP  2;  Calculate 


.  as 


follows . 


[7.12] 


A. 


A„„  =  0.3855216  +  0.830581  +  1.789433  -  3.005536 


(d)  The  consistency  index  is  then  calculated  as 


^  ^  _  (3.00536-3) 
2 


0.002768 


using  [7.09]. 


[7.13] 
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The  denominator  of  the  consistency  ratio  is  the  random 
cc-.sistency .  That  random  value  is  specified  by  the  number  of 
elements  or  activities  for  a  given  matrix  according  to  a  random 
index  table.  The  random  index  table  used  in  this  study  to 
determine  the  random  consistency  was  generated  at  the  Oak  Ridge 
National  Laboratory:^ 


,  0. 002768 

0.58 


0.0047724 


NOTE:  R.I.=  0.58  (this  random  index  value  is  obtained 
from  Saaty's  text). 

The  consistency  ratio  for  the  A3  mix  is  less  than  the  10  percent 
Saaty  uses  to  rate  consistency  in  the  pairwise  comparison  matrix 
and  indicates  the  decisionmaker's  values  did  not  dramatically 
change  during  the  pairwise  rating  process. 

d.  Subject  Matter  Experts  and  Weapon  System 
Scoring.  A  second  methodology  was  used  to  obtain  the  subjec¬ 
tive  judgments  of  subject  matter  experts  (Modernization  Program 
Execution  Group  (PEG])  on  the  relative  value  of  23  weapon 
systems  with  respect  to  the  SIAM  factors.  Experts  were  asked  to 
rate  each  of  the  weapon  systems  on  the  rating  instruments  based 
on  a  standard  scale  from  1  to  9  (see  Figure  7-5).  The  scores 
given  by  each  expert  were  then  summed  to  determine  the  combined 
score  for  each  of  the  23  systems.  The  results  of  this  analysis 
confirmed  that  key  air  defense  and  assault  helicopter  weapon 
systems  were  viewed  as  more  versatile,  deployable,  and  more 
critically  needed. 


Mbid.,  Page  21. 
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TERMINALLY  GUIDED  WEAPON  (TGW)  SCORING 


POLITICAL  RISK:CONGRESSIONAL  OPINION 


DEFINITON: 


SCORES: 


The  congressional  opinion  measure  of  political 
risk  is  a  subjective  evaluation  of  the  attitudes  of  the 
congress  toward  a  proposed  program.  It  ranges  from 
significant  positive  support  for  the  program  to  significant 
opposition  to  the  program. 

□□nDDDnnn 

1  2  3  4  5  4  7  !  ! 


■Igalflcaat 

oppoaltion 

(bad) 


•adioa 

auppert 


poaltiva 

auppert 

( good  I 


Figure  7-6.  Example  Scoring  Instrument 


(1)  An  initial  scoring  instrument  was  developed  using 
literature  reviews,  brainstorming,  and  expert  opinion.  This 
excunple  instrument  was  validated  using  senior  staff  officers 
within  the  Department  of  the  Army  Staff. 


(2)  A  final  scoring  instrument  was  constructed  using  the 
results  of  the  validation  process.  The  study  sponsor  assisted 
the  team  in  selecting  a  final  panel  for  conducting  the  weapon 
system  scoring  based  on  their  expertise  and  positions  on  the 
Army  Staff. 


(3)  The  solicitation  of  scores  from  this  final  panel  was 
conducted  in  a  two-step  process.  The  first  step  consisted  of 
providing  each  scorer  a  complete  set  of  scoring  instruments. 

The  experts  were  asked  to  use  appropriate  documents,  such  as 
media  articles.  Congressional  reports,  technical  papers,  COEAs, 
CBRS,  and  Modernization  plans  etc.,  to  develop  a  knowledge  base 
for  each  of  the  systems  (to  be  used  during  the  scoring  process). 
The  second  step  consisted  of  a  deskside  briefing  to  ensure  the 
respondents  understood  and  felt  comfortable  with  the  Value  Added 
methodology  and  approach. 


(4)  The  following  is  an  illustrative  example  of  how  the 
scoring  would  be  completed.  Table  7-2  is  an  example  of  a 
tradeoff  issue  in  which  the  decisionmaker  is  seeking  the  most 
cost  effective  mix  of  armor  and  antiarmor  systems. 


Table  7-2.  Example  Scoring  Matrix 


PRCO* 

AV/D*  * 

it 

OTR 

WEIGHTS 

0.1282705 

0.2763505 

0.595378 

BASE  CASE 

31 

19 

36 
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ALT  1 

29 

36 

27 

ALT  2 

16 

24 

17 

*  -  Political  Risk:  Congressional  Opinion 
-  Asset  Versatility  and  Deployablility 
»»•  -  Operational  and  Technical  Risk 


Figure  7-6 
experts  as 


suiwnarizes  the  scoring  for  five  subject  matter 
calculated  using  the  following  steps; 
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BC 

Altl 

Alt  2 

PRCO 

31 

29 

16 

AVID 

19 

36 

24 

OTR 

36 

27 

17 

Figure  7-7.  Illustrative  Example  of  Summary  Scores 
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e.  Analysis.  The  results  of  the  analysis  associated  with 
this  portion  of  the  method  are  contained  in  Appendix  E.  The 
study  team  wanted  to  keep  the  main  body  of  the  report 
unclassified. 

(1)  The  methodology  used  in  this  study  to  help  the 
decisionmaker  determine  the  most  desirable  RDA  portion  of  the 
POM  to  meet  the  program  year's  requirements  can  be  used  in  other 
areas  as  well.  It  can  be  used  for  any  decision  with  multiple 
criteria  to  evaluate  the  relative  strength  of  the  variables. 

The  methodology  can  be  used  to  compare  alternative  weapon 
systems,  deployment  options,  training  devices,  etc.  Conse¬ 
quently,  the  methodology  can  aid  the  decisionmaker  in  ensuring 
that  a  force  is  properly  resourced.  The  flexibility  and  sim¬ 
plicity  of  the  Analytic  Hierarchy  Process  allow  the  decision¬ 
maker  to  better  understand  complex  systems  of  interrelated 
components,  such  as  resources,  desired  Outcomes,  or  objectives. 

(2)  The  analysis  shows  the  decisionmakers  believe  that, 
of  the  major  criteria,  the  combat  objective  holds  the  most 
relative  importance;  the  most  relatively  important  combat  sub¬ 
objective  is  correlation  of  forces  and  means;  while  the  four 
most  relatively  important  SIAM  subobjectives  are  asset  versa¬ 
tility  and  deployability,  criticality  of  need  as  related  to 
current  capability,  political  risk  with  respect  to  Congressional 
opinion,  and  operational  and  technical  risk.  Judgments  were  not 
taken  on  the  noncombat  subobjectives.  As  a  result  of  feedback 
received  from  the  senior  leadership  during  the  conduct  of  the 
survey,  this  area  was  expanded  and  will  be  more  thoroughly 
evaluated  during  Value  Added  Phase  II. 

7-3.  SUMMARY.  The  development  of  the  Value  Added  methodology 
specifically  included  two  components  (explicit  and  implicit)  to 
the  valuation  portion  of  VAA.  The  implicit  effectiveness 
component  was  specifically  designed  to  measure  the  subjective 
element  of  value.  This  measurement  of  the  subjective  element  of 
decisionmaking  was  included  in  the  VAA  methodology  based  on  the 
findings  of  previous  work  done  in  this  area  for  both  the  Army 
and  others.  One  of  the  key  findings  of  our  literature  review 
stressed  the  importance  of  including  the  secondary  factors  which 
impact  upon  decisions.  The  methodology,  as  currently  envi¬ 
sioned,  uses  an  approach  that  is  based  on  the  Analytical 
Hierarchy  Process  and  appears  to  be  very  useful  in  helping 
decisionmakers  focus  their  attitudes  and  identify  the  factors 
which  impact  upon  PPBES  programming  decisions.  The  results  of 
this  module  are  passed  to  the  Effectiveness  Integration  Module 
in  order  to  combine  their  effects  with  those  of  the  Explicit 
Effectiveness  Module. 
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CHAPTER  8 

EFFECTIVENESS  INTEGRATION 


8-1.  INTRODUCTION 

a.  The  purpose  of  this  chapter  is  to  provide  a  discussion  of  the 
integration  of  multiple  measures  of  effectiveness  into  a  single 
measure.  This  chapter  also  discusses  the  implementation  of  a  Value 
Added  Analysis  tool  for  this  function 

(1)  Introduction 

(2)  Develop  Context  for  Effectiveness  Integration 

(3)  Select  and  Develop  Appropriate  Models  and  Tools 

(4)  Implement  Models  and  Tools 

(5)  Summary 

b.  Figure  8-1  depicts  the  Effectiveness  Integration  Module 
and  how  it  relates  to  the  overall  VAA  methodology.  The  Effec¬ 
tiveness  Integration  Module  is  the  section  of  the  VAA  method¬ 
ology  which  produces  the  single  measure  of  value  associated  with 
VAA  and  is  used  to  conduct  a  cost  effectiveness  evaluation.  The 
output  of  this  module  consists  of  program  value  coefficients  for 
each  of  the  alternatives  being  investigated. 


Figure  8-1.  VAA  Methodology  -  Effectiveness  Integration 

Overview 
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c.  The  Value  Added  methodology  recognizes  both  the 
advantages  and  limitations  of  combining  multiple  measures  of 
effectiveness,  and  consequently  allows  decision  analysis  either 
in  an  integrated  mode  or  in  a  mode  which  only  displays  the 
measures  of  effectiveness  in  tabular  form.  Thus,  this  chapter 
can  be  viewed  as  a  discussion  of  an  optional  part  of  the 
methodology.  In  practice,  however,  because  of  the  insights 
gained  from  integrated  measures  of  effectiveness,  and  because  of 
the  input  requirements  of  "downstream"  tools  such  as  FOMOA, 
integration  has  almost  always  been  performed.  In  fact,  the 
outputs  of  the  tool  discussed  in  this  chapter  provide  a 
meaningful  analysis  product  in  their  own  right  for  comparing 
alternatives . 

d.  The  analytic  tool  selected  for  measure  of  effectiveness 
integration  is  the  Technique  for  Order  Preference  by  Similarity 
to  an  Ideal  Solution,  better  known  by  its  acronym,  TOPSIS.  This 
technique  was  developed  by  Ching-Lai  Hwang^  and  Kwangsun  Yoon  at 
Kansas  State  University  and  has  been  widely  used  by  the  Army  and 
taught  in  operations  research  classes  at  the  Army  Logistics 
Management  Center.  It  is  based  upon  the  concept  of  choosing 
alternatives  which  lie  in  an  n-dimensional  (sea  Figure  8-2) 
Euclidean  space  closest  to  a  theoretically  ideal  position  and 
farthest  from  a  theoretically  worst  position. 


e.  Computer  software  to  implement  TOPSIS  was  the  first  tool 
designed  and  implemented  to  operate  in  the  capsule  environment 


^ Hwang,  Ching-Lai  and  Yoon,  Kwangusn,  Multiple  Attribute  Decision  Making. 
M£.tlTLQds _ and  Applications.  Springer-Verlag,  Berlin,  1981. 
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of  the  data  base  computer  chosen  for  the  Value  Added  prototype. 
While  software  to  compute  TOPSIS  coefficients  was  available 
previously,  the  Value  Added  team  designed  and  wrote  a  TOPSIS 
tool  from  scratch  in  order  to  ensure  wide  applicability  of  the 
methodology  and  to  overcome  inherent  problem  size  limitations  of 
the  existing  software.  The  TOPSIS  software  described  was 
written  in  ANSI  standard  C  language  and  has  been  successfully 
compiled  and  used  on  MS-DOS  and  Macintosh  microcomputers  and 
UNIX  workstations.  Because  TOPSIS  is  useful  in  its  own  right,  a 
standalone  version  has  been  implemented  and  extensively  tested 
as  part  of  this  study.  A  brief  set  of  instructions  for  use  of 
the  stand  alone  TOPSIS  software  in  the  MS-DOS  environment  is 
provided  in  Appendix  G.  Instructions  for  use  of  TOPSIS  and 
other  Value  Added  Analysis  tools  in  the  METAPHOR  environment 
will  be  provided  as  part  of  Phase  II  of  the  Value  Added  project. 

8-2.  DEVELOP  CONTEXT  FOR  EFFECTIVENESS  INTEGRATION 

a.  Nature  of  the  Problem.  Techniques  for  integration 
of  multiple  decision  criteria,  such  as  TOPSIS,  are  well  suited 
to  the  type  of  management  decision  problem  typified  by  Value 
Added  Analysis.  These  problem  characteristics  include: 

(1)  A  requirement  to  build  a  rank-ordered  preference 
list  of  candidate  systems  from  among  a  relatively  constrained 
set  of  initial  alternatives. 

(2)  A  requirement  to  be  able  to  compare  the  relative 
strength  of  preference  (cardinal  rank)  among  the  candidates. 

(3)  A  set  of  quantifiable  criteria  related  to  attributes 
associated  with  each  alternative. 

(4)  A  set  of  weights  which  express  the  relative  measure 
of  importance  the  decision  authority  attaches  to  each  criteria. 

(5)  A  modicum  of  confidence  in  the  statistical 
independence  and  reliability  of  the  measures  associated  with  the 
different  criteria.  This  confidence  may  be  "subjective"  based 
upon  the  decision  authority's  judgment. 

(6)  A  requirement  for  sensitivity  measures  associated 
with  variance  in  input  values  or  decision  authority  tolerance  to 
error. 


b.  Operational  Considerations.  Operational  constraints 
require  the  use  of  a  measure  of  effectiveness  integration  tool. 
Interviews  with  sponsors  and  review  of  decision  science 
literature  substantiated  the  following  operational 
characteristics : 

(1)  Real-world  problems  contain  too  many  alternatives  (n 
>  50)  and  too  many  decision  criteria  (n  >  10)  for  human 
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comprehension  in  tabular  form.  Some  simplification  procedure  is 
required. 

(2)  Available  optimization  techniques  (e.g.,  linear  and 
dynamic  programming)  require  objectively  quantified  and 
integrated  measures  of  effectiveness.  In  particular,  even 
techniques  which  deal  with  multiple  measures  of  effectiveness 
(e.g.,  goal  programming)  always  contain  either  an  explicit  or 
implicit  combining  function. 

(3)  Even  when  mathematically  precise  measures  of 
effectiveness  are  not  available  or  integration  of  available 
measures  appears  infeasible,  operational  estimates  must  be  made 
with  sufficient  precision  to  allow  meaningful  analysis  to 
proceed. 


c.  Problem  Domain.  The  problem  domain  (prioritized 
allocation  of  resources  in  construction  of  the  Army  POM)  is 
suited  to  the  use  of  a  measure  of  effectiveness  integration 
tool.  Important  characteristics  include: 

(1)  A  hierarchical  schema  for  relating  measures  of 
effectiveness  has  been  defined  (see  Chapter  7). 

(2)  Integration  tools  of  this  type  (including  TOPSIS 
itself)  have  been  previously  and  are  currently  being  used  with 
varying  degrees  of  success  in  this  domain.  In  particular, 

TRADOC  has  extensive  experience  in  using  the  TOPSIS  methodology 
to  construct  the  Field  Long-Range  Research  and  Development  Plan. 

(3)  Key  decision  authorities  are  knowledgeable  con¬ 
cerning  the  underlying  assumptions,  capabilities  and  limitations 
of  this  approach  in  this  problem  domain. 

8-3.  SELECT  AND  DEVELOP  APPROPRIATE  ANALYTICAL  MODELS 
AND  TOOLS 

a.  Criteria.  The  following  criteria  led  to  the  selection 
of  the  TOPSIS  tool: 

(1)  The  chosen  integration  technique  must  be  known  and 
accepted  within  the  decision  science  literature. 

(2)  The  chosen  integration  technique  must  be  capable  of 
being  implemented  and  used  within  the  time  constraints 
available . 

(3)  The  chosen  integration  technique  must  not  have  data 
requirements  that  exceed  the  collection  capability  of  the 
analysis  effort. 

(4)  The  chosen  integration  technique  must  produce 
outputs  compatible  with  other  Value  Added  Analysis  tools, 
especially  the  Force  Modernization  Analyzer. 
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b.  Review  and  Evaluate  Alternatives.  Several 
alternatives  were  extensively  considered  for  use  in  lieu  of 
TOPSIS.  They  were  found  to  be  unsuitable  for  the  following 
reasons : 

(1)  Multiple  attribute  utility  functions  were  not 
accepted  because  of  the  lack  of  consensus  on  the  definition  of 
the  output  measure  ("utils").  Utility  measures  (based  on 
consumer  preference  theory)  have  the  most  extensive  theoretical 
literature  background,  but  they  are  also  much  more  difficult  to 
develop,  implement,  and  document. 

(2)  Probability-based  decision  trees  were  not  chosen 
because  of  the  difficulty  of  collecting  data  related  to  the 
attributes  and  probabilities  required  for  such  an  analysis.  An 
extensive  case  study  related  to  multiple  theaters  of  conflict 
and  multiple  levels  of  conflict  intensity  was  prepared,  and 
showed  potential  because  it  made  a  direct  link  to  the  Army 
strategic  planning  process.  However,  as  the  methodology 
developed,  the  decision  tree  approach  was  later  abandoned 
because  of  an  inability  to  properly  define  all  of  the  decision 
paths  and  alternatives.  A  by-product  of  this  effort  was  that 
Value  Added  Analysis  beceune  much  more  focused  toward  weapon 
system  acquisition,  wherein  alternatives  are  more  clearly 
defined, 

(3)  Other  methods  of  combination  (e.g.,  count  of  advan¬ 
tages  and  disadvantages)  were  considered  but  rejected  because  - 
they  either  did  not  produce  the  desired  cardinal  preference 
weight  or  because  they  were  relatively  unknown  to  the  decision 
authorities . 

8-4.  IMPLEMENT  ANALYTICAL  MODELS  AND  TOOLS 

a.  Notation.  The  following  notation  is  used  in  the 
TOPSIS  methodology. 
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Aj  -  the  alternative  considered, 

Xjj  -  the  numerical  outcome  of  the  i^  alternative  with  respect  to  the  f"*  aiterion, 

R  -  normalized  decision  matrix, 

ry  -  an  element  of  the  normalized  decision  matrix  with  respect  to, 
the  i^^  alternative  and  aiterion, 
m  -  total  number  of  alternatives, 
n  -  total  number  of  attributes, 

Wj  -  weight  assodated  with  the  aiterion, 

1/  -  weighted  normalized  dedsion  matrix, 

A*  -  ideal  artifidal  alternative  (  most  preferable  alternative), 

A“  -  negative-ideal  artifidal  alternative  (least  preferable  alternative), 

Vy  -  an  element  of  the  weighted  normalized  dedsion  matrix  v;ith  respect  to, 
the  i^^  alternative  and  aiterion, 

J  -  (i  e  { 1 ,2 I  j  assodated  with  benefit}, 

J  -  {j  £  { 1 2 . n}  I  j  assodated  with  cost}, 

Sj*  -  separation  of  the  i“^  alternative  from  A*, 

Sp  -  separation  of  the  i‘^  alternative  from  A”, 

C|*  -  relative  doseness  of  A,  with  respect  to  A*. 

b.  The  Algorithm.  The  algorithm  is  from  Hwang  and  Yoon,^ 
cited  previously.  A  description  is  provided  below: 

STEP  1:  Construct  a  normalized  decision  matrix,  R; 


^  Hwang,  Ching-Lai  and  Yoon,  Kwangsun,  Multiple  Attribute  Decision  Making. 
Methods  and  Applications.  Springer-Verlag,  Berlin,  1981.  Pages  128  through  140. 
The  algorithms  and  formula  listed  here  were  obtained  directly  from  the  above 
referenced  pages  of  Hwang  and  Yoons  text. 
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STEP  2:  Construct  the  weighted  normalized  decision  matrix,  V; 
such  that 

[8.02] 


'^11 

V  = 

STEP  3:  Determine  ideal  and  negative-ideal  solutions;  Let  the 
two  artificial  alternatives  A  and  A~  be  defined  as: 

[8.03] 

A*  -  .  (max  v,j  j  eJ),  (min,  vjGJ)|i  - 


A'  -  .  (min  j  G/),  (max  Vi-|G/)|i  - 


STEP  4:  Calculate  the  separation  measure.  The  separation 
between  each  alternative  can  be  measured  by  the  n-dimensional 
Euclidean  distance.  The  separation  of  each  alternative  from  the 
ideal  alternative  is  given  by: 

[8.04] 


Similarly,  the  separation  from  the  negative-ideal  alternative  is 
given  by: 

[8.05] 


STEP  5:  Calculate  the  relative  closeness  to  the  ideal 
solution. 


STEP  6:  Rank  the  preference  order. 

STEP  7 :  Perform  sensitivity  analysis  and  graphically  portray 
results . 
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c.  The  Implementation.  A  C  language  computer  software 
program  was  designed  and  written  to  support  the  Value  Added 
Analysis  effort.  This  program  has  the  following  modules; 

(1)  General  Purpose  Math  Support  Routine  Module. 

This  module  contains  memory  allocation  routines  and  numeric 
conversion  routines  that  are  broadly  applicable  to  all  of  the 
Value  Added  Analysis  support  software.  The  support  routines  are 
based  upon  similar  routines  contained  in  Nximerical  Recipes  in  C 
by  Press,  Flannery,  Teukolsky,  and  Vettering^. 

(2)  TOPSIS  Matrix  Input-Output  Routine  Module. 

This  module  contains  special  purpose  routines  to  read  in  the 
TOPSIS  input  file  and  allocate  memory  for  the  TOPSIS  matrix.  Of 
special  importance  are  the  procedures  which  determine  the  number 
of  rows  and  columns  in  the  matrix.  The  software  is  constructed 
so  that  the  size  of  the  TOPSIS  matrix  is  not  predetermined  and 
works  with  large  sizes  limited  only  by  available  memory.  On  a 
Macintosh  microcomputer,  matrices  of  sizes  up  to  120  rows  and 
columns  have  been  tested.  Considering  that  each  floating  point 
number  takes  8  bytes,  matrices  with  hundreds  of  thousands  of 
cells  are  possible.  Appendix  G  contains  exeunple  inputs  and 
outputs  using  the  A3  illustrative  example. 

(3)  TOPSIS  Calculation  Roac^ne  Module.  This  module 
actually  encodes  the  TOPSIS  algorithm  as  described  by  Hwang  and 
Yoon.  All  TOPSIS  specific  calculation  is  performed  in  this 
module . 

(4)  Main  Program  Module.  This  module  contains  the 
top  level  routine  ("main")  and  calls  all  other  modules.  Its 
primary  purpose  is  to  parse  the  command  line  interface  arguments 
and  open  the  appropriate  input  and  output  files. 

8-4.  SUMMARY.  The  closeness  value  becomes  the  program  value 
coefficient  and  is  the  single  measure  of  benefit  used  in  the 
follow-on  modules.  This  value  is  used  in  the  optimization 
module  as  a  means  of  calculating  the  VAA  cost  effectiveness. 

The  closeness  value  can  also  be  used  to  rank  order  alternatives 
based  solely  on  the  criterion  used  in  the  TOPSIS  calculations. 
This  capability  provides  us  with  a  flexibility  to  stop  the 
analysis  at  this  juncture  or  continue  based  on  the  needs  of  the 
decisionmaker. 


^  Press;  Flannery;  Teukolsky;  and  Vettering,  Numerical  Recipes  in  C.  Oxford 
University  Press,  1988. 
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CHAPTER  9 

COST  EFFECTIVENESS  ANALYSIT 


9-1.  INTRODUCTION 

a.  The  purpose  of  this  chapter  is  to  provide  a  discussion 
of  the  optimization  tool  used  to  produce  an  acquisition  strategy 
given  costs  and  relative  measures  of  benefit.  This  chapter  also 
discusses  the  implementation  of  a  Value  Added  Analysis  tool  for 
this  funccion. 

(1)  Introduction 

(2)  Develop  Context  for  Acquisition  Strategy 

(3)  Select  and  Develop  Appropriate  Models  &  Tools 

(4)  Implement  Models  and  Tools 

(5)  Summary 

b.  Figure  9-1  depicts  the  Optimization  Module  and  how  it 
relates  to  the  overall  VAA  methodology.  The  Optimization  Module 
is  the  section  of  the  VAA  methodology  that  is  used  to  conduct 
the  cost  effectiveness  evaluation.  The  module  allows  us  to 
simultaneously  trade  off  alternatives  based  on  their  cost  and 
effectiveness  and  also  to  schedule  the  alternatives  acquisition. 
The  output  of  this  module  consists  of  a  funding  and  acquisition 
stream  very  similar  to  the  streams  found  in  the  Ar  POM. 
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(1)  The  Value  Added  methodology  does  not  require  the 
production  of  a  procurement  strategy.  However,  the  optimization 
tool  does  provide  the  integration  of  costs  and  benefits  into  a 
comprehensive  cost-benefit  analysis.  Thus,  this  chapter  can  be 
viewed  as  a  discussion  of  an  optional  part  of  the  methodology, 
but  in  practice,  an  acquisition  strategy  and  cost  benefit 
analysis  has  almost  always  been  performed. 

(2)  The  analytic  tool  selected  for  development  of  the 
acquisition  strategy  was  the  Force  Modernization  Analyzer, 
better  known  by  its  acronym,  FOMOA.  This  technique  was 
developed  by  Robert  Schwabauer^  at  the  US  Army  Concepts  Analysis 
Agency  and  has  been  widely  used  by  the  Army  for  the  production 
of  weapon  system  modernization  plans.  It  is  based  upon  the 
concept  of  linear  progreunming  and  econometric  modeling  of  cost- 
quantity  relationships.  The  Value  Added  FOMOA  formulation 
considers  production,  total  obligation  authority  (TOA),  and 
force  structure  constraints  in  conjunction  with  cost  and  benefit 
data  in  order  to  determine  an  optimal  mix  of  weapon  system 
procurements  by  year . 

(3)  Computer  software  to  implement  FOMOA  for  Value  Added 
Analysis  differs  from  that  described  by  Schwabauer.  For  Value 
Added  Analysis,  a  tool  was  designed  and  implemented  to  operate 
in  the  capsule  environment  of  the  data  base  computer  chosen  for 
the  Value  Added  prototype.  The  more  traditional  FOMOA  operates 
using  commercially  procured  software  only  in  the  environment  of 
an  Apple  Macintosh  microcomputer  spreadsheet.  As  such,  the 
design  of  the  FOMOA  software  reflects  the  "icon-and-arrow"  data 
base  technology  discussed  by  Cunningham, 2  et  al.  The  FOMOA 
software  described  was  written  in  ANSI  standard  C  language  and 
has  been  successfully  compiled  and  used  on  MS-DOS  and  Macintosh 
microcomputers  and  UNIX  workstations.  For  the  Value  Added 
Analysis  Project,  Dr.  Schwabauer  prepared  an  original  algebraic 
formulation  of  the  weapon  systems  acquisition  strategy  problem, 
which  was  independent  of  the  specific  weapon  system  costs  and 
parameters  involved.  A  total  separation  of  problem  formulation 
and  data  was  thereby  achieved.  Because- FOMOA  is  useful  in  its 
own  right,  a  standalone  version  has  been  implemented  and 
extensively  tested  as  part  of  this  study.  A  brief  set  of 
instructions  for  use  of  the  standalone  FOMOA  software  in  the 
Macintosh  environment  is  provided  at  the  end  of  this  chapter. 
Instructions  for  use  of  FOMOA  and  other  Value  Added  Analysis 
tools  in  the  environment  of  the  data  base  machine  will  be 
provided  as  part  of  Phase  II  of  the  Value  Added  project. 


^  Schwabauer,  Robert,  "User's  Manual  for  Force  Modernization  Analyzer  (FOMOA) 
Level  1,"  US  Army  Concepts  Analysis  Agency  Report  CAA-D-89-3,  Bethesda,  MD. 
November  1989. 

2  Cunningham,  Van;  Koury,  Robert;  and  Richmann,  James,  "Less  Expensive,  More 
Powerful  Database  Analysis,"  Database  90  Conference,  US  Navy  Automation 
Systems  Command,  San  Diego  CA,  June  1990. 
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9-2.  DEVELOP  CONTEXT  FOR  ACQUISITION  STRATEGY 
DEVELOPMENT 

a.  Nature  of  the  Problem.  Techniques  for  development 
of  acquisition  strategies  such  as  FOMOA’ are  well  suited  to  the 
type  of  management  decision  problem  typified  by  Value  Added 
Analysis.  These  problem  characteristics  include: 

(1)  A  requirement  to  project  procurements  by  dollars  and 
quantities  for  candidate  systems  by  year. 

(2)  A  requirement  to  be  able  to  consider  the  effects  of 
changing  production  and  TOA  constraints  on  the  weapon  mix. 
Changes  to  constraints  will  be  made  frequently  and  results  must 
be  obtained  on  a  time  sensitive  basis.  Responsiveness  is  more 
important  than  precision  for  this  requirement. 

(3)  A  set  of  quantifiable  attributes  that  can  be 
associated  with  each  candidate  system  including  cost,  an 
integrated  measure  of  benefit,  production  constraints,  and  force 
structure  requirements. 

(4)  A  requirement  to  determine  sensitivity  measures 
associated  with  variance  in  input  values  or  decision  authority 
tolerance  to  error. 

b.  Operational  Considerations.  Operational  constraints 
require  the  use  of  an  acquisition  strategy  tool.  Interviews 
with  sponsor  management  and  review  of  decision  science  liter¬ 
ature  substantiated  the  following  operational  characteristics: 

(1)  Value  Added  acquisition  strategy  problems  contain 
multiple  alternatives  (n  >  50)  and  multiple  types  of  constraints 
(n  >  10).  Some  simplification  procedure  is  required.  Also,  the 
preferred  solution  is  usually  a  mix  of  systems  rather  than  the 
selection  of  only  one  system  for  procurement. 

(2)  All  optimization  techniques  (e.g.,  linear  and 
dynamic  programming)  require  extensive  analysis  during  the 
formulation  stage  in  order  to  tailor  the  algebraic  specification 
to  the  problem  at  hand.  Typically,  this  formulation  is  provided 
as  data  coefficients  which  form  the  input  to  the  mathematical 
optimization  tool.  This  approach  usually  requires  long  time 
periods  to  collect,  validate,  and  prepare  data  inputs.  An 
ability  to  generalize  the  formulation  so  that  data  can  be 
obtained  automatically  from  a  data  base  query  would  reduce 
errors  in  transcription  and  formulation. 

(3)  Even  when  mathematically  precise  cost  and  effec¬ 
tiveness  data  are  not  available  or  optimization  of  the  acqui¬ 
sition  strategy  appears  infeasible,  operational  estimates  and 
preliminary  plans  must  be  made  with  sufficient  precision  to 
allow  programming  and  budgeting  to  proceed. 
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(4)  All  programming  and  budgeting  tradeoffs  must  accom¬ 
modate  multiple  and  sometimes  conflicting  goals.  The  acquisi¬ 
tion  strategy  tool  must  be  able  to  optimize  the  mix  of  systems 
while  considering  these  multiple  goals.  The  FOMOA  formulation 
developed  during  Phase  I  of  Value  Added  Analysis  does  not  ade¬ 
quately  address  multiple  goals  simultaneously.  In  order  to 
consider  multiple  goals,  separate  formulations  must  be 
constructed  and  run. 

9-3.  SELECT  AND  DEVELOP  APPROPRIATE  ANALYTICAL  MODELS 
AND  TOOLS 

a.  Criteria.  The  following  criteria  led  to  the  selection 
of  the  FOMOA  tool: 

(1)  The  chosen  optimization  technique  must  be  known  and 
accepted  within  the  decision  science  literature. 

(2)  The  chosen  optimization  technique  must  be  capable  of 
being  implemented  and  used  within  the  time  constraints 
available. 

(3)  The  chosen  optimization  technique  must  not  have  data 
requirements  that  exceed  the  collection  capability  of  the 
analysis  effort. 

(4)  The  chosen  optimization  technique  must  address 
essential  constraints  (e.g.,  production  limits  by  year  and  force 
structure  requirement  maximums  by  year).  Without  these  con¬ 
straints,  no  acquisition  strategy  would  be  considered  valid. 

(5)  The  chosen  optimization  technique  must  be  capable  of 
being  used  by  personnel  with  a  wide  variety  of  backgrounds  and 
educational  experiences.  This  objective  was  only  partially 
achieved  during  Phase  I  of  Value  Added  Analysis  and  is  a 
priority  requirement  for  Phase  II. 

(6)  The  chosen  optimization  technique  must  produce 
outputs  required  by  the  study  sponsor.  In  particular,  the  tool 
must  produce  a  time-phased  acquisition  plan  in  a  format 
recognizable  as  input  to  the  Army  POM. 

b.  Review  and  Evaluate  Alternatives.  Several  alter¬ 
natives  were  extensively  considered  for  use  in  lieu  of  FOMOA. 
None  were  selected  for  the  following  reasons: 
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(1)  The  use  of  a  Leontif^  Input-Output  Model  as 
described  in  the  Army  Dollar  Resource  Study2  was  not  accepted 
because  leadership  viewed  the  technique  as  a  "black  box,"  too 
complex  for  management  understanding  of  the  effects  of  changing 
inputs.  Also,  the  model  developed  for  the  Army  Dollar  Resource 
Study  was  memoryless  in  that  previous  decisions  were  easily 
changed  during  subsequent  analysis.  Management  considered  it 
important  that  decisions  made  in  the  past,  and  in  the  process  of 
implementation,  should  not  be  revisited.  Stability  is  an 
important  criterion. 

(2)  The  use  of  a  capital  budgeting  technique  as 
described  by  Michael  Anderson, 3  which  uses  vectors  of  cost- 
benefit  swapped  in  and  out  of  a  proposed  solution  set  based  upon 
TOA  and  relative  cost-benefit  comparisons  was  not  used  because 
the  technique  was  relatively  inflexible  in  accounting  for 
multiple  measures  of  benefit.  In  particular,  the  technique  did 
not  adequately  address  subjective  measures  of  value. 

(3)  The  use  of  simple  rank  order  preference  was  con¬ 
sidered  but  not  used  because  it  did  not  analytically  integrate 
all  appropriate  information  in  arriving  at  an  acquisition 
strategy.  It  should  be  noted  that  simple  rank  order  preference 
is  the  technique  currently  used  to  construct  the  Army  POM.  It 
was  the  strong  opinion  of  the  study  sponsor  that  the  existing 
method  was  not  sufficiently  analytic  and  resulted  in  competition 
among  proponents  based  upon  suboptimal  decision  criteria. 

9-4.  IMPLEMENT  ANALYTICAL  MODELS  AMD  TOOLS 

a.  Formulation.  The  formulation  of  the  optimization 
problem  solved  in  this  module  can  be  summarized  as  follows . 

(1)  Objective.  The  objective  is  to  maximize  the  total 
value  added  by  the  systems  in  the  force  over  the  planning 
horizon.  The  objective  coefficients  are  the  value  added 
coefficients  computed  in  TOPSIS.  These  coefficients  are  given 
as  a  function  of  time  period  and  equipment  system. 

(2)  Constraints.  There  are  three  classes  of 
constraints.  The  first  class  consists  of  funding  constraints 
which  limit  expenditures  for  procurement  to  be  no  greater  than 
the  total  obligation  authority  for  those  systems.  The  second 
set  of  constraints  deals  with  force  structure.  A  minimum  and 
maximum  number  of  each  system,  based  on  force  structure 


1  Leontif,  Wassily  W.  "The  World  Economy  of  the  Year  2000,"  Scientific 
American,  September  1980. 

“  Army  Dollar  Resource  Allocation  II  Study.  US  Army  Concepts  Analysis  Agencv. 
1978. 

^  Anderson,  Michael,  "Capital  Budgeting  White  Paper,"  •  US  Army  Training  and 
Doctrine  Command,  unpublished  paper,  1989. 
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requirements,  is  specified,  constraining  the  number  of  system  of 
each  type  in  the  force.  The  final  class  of  constraints  deals 
with  production  limitations.  A  minimum  production  quantity 
might  be  specified  to  ensure  an  economic  quantity  is  produced. 
The  maximum  production  capacity  is  addressed  by  constraining  the 
total  number  of  systems  of  each  type  that  can  be  produced. 

(3)  Decision  Variables.  The  answer  to  the 
optimization  problem  consists  of  the  number  of  each  type  of 
system  produced  during  each  time  period.  For  the  purpose  of 
Value  Added  Analysis,  these  time  periods  will  be  fiscal  years 
that  are  included  in  the  POM  and  EPA. 

(4)  Notation.  The  following  notation  is  defined  for 
the  Value  Added  version  of  the  FOMOA  Model.  This  formulation 
and  notation  were  baaed  on  the  original  work  developed  at  CAA 
(see  Appendix  H,  paragraph  H-2). 
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ajj  -  value  added  coefficient  for  system  i  in  time  period  j, 

Xjj  »  number  of  system  i  produced  in  time  period  j, 

Xjj  -  number  of  systems  i  produced  in  time  period  j 
without  considering  production  line  status, 

Mj  -  total  obligation  authority  in  time  period  j, 

S|  -  inherited  assets  of  system  i  available  in  year  1 , 
fjk  -  programed  fielding  in  year  k  for  system  i  where, 
k  -  1  denotes  2  years  before  planning  period, 
and  k  -  2  denotes  1  year  before  planning  period, 
yij  -  number  of  systems  retired  in  time  period  j, 

Ojj  -  number  of  systems  on-hand  in  time  period  j, 

Cpjj  -  procurement  cost  of  system  i  in  time  period  j, 

Cojj  -  O&S  cost  of  system  i  in  time  period  j, 
eijj  -  lower  limit  on  number  of  system  i  required  in  time  period  j, 
euij  -  upper  limit  on  number  of  system  i  required  in  time  period  j, 
X(3ij  -  lower  production  limit  on  number  of  system  i  in  time  period  j, 
Xtjj  -  upper  production  limit  on  number  of  system  i  in  time  period  j. 
Pi,  -  production  line  status  in  time  period  j  for  system  i, 
where  py  -  1  if  line  is  open,  and  0  otherwise, 

I  »  set  of  all  systems, 

J  -  set  of  all  time  periods. 


(5)  Dependent  Relationships  Among  Variables.  The 

following  relation  is  defined  for  determining  the  number  of 
systems  onhand  during  each  time  period. 
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©ii  -  Si+fij-yii.  Vi  el. 

©12  -  Si+fij-yi2.  Viel. 
ey-ej(j_i)+Xi(j_2)-yij.  Viel,  Vj>2. 


Note  here  that  in  order  to  take  into  consideration  the 
status  of  the  production  lines,  we  must  perform  the  following 
preprocessing  step. 

[9.02] 


Xij  if  Pi|-  1, 


0  Otherwise. 


Thus,  Xij  is  constrained  to  be  zero  when  the  production 
line  is  not  open  in  particular  time  period  j  for  system  i. 

(6)  Objective  Function.  The  following  objective 
function  maximizes  the  total  "Value  Added"  by  the  procured 
systems  over  the  entire  planning  horizon. 

[9.03] 


I'l  Ml 

(7)  Constraints.  As  discussed  above,  there  are  three 
classes  of  constraints.  The  first  class  involves  funding 
constraints . 


[9,04  ] 


ijXjj  s  Mj,  V  j  S  J. 


Notice  that  we  consider  only  procurement  costs  here. 

O&S  costs  are  used  in  the  prototype  for  accounting  purposes 
only.  Notice  also  that  cost  values  used  here  constitute  average 
unit  costs.  Costs  are  actually  a  function  of  the  quantity 
produced  (see  chapter  6).  No  provision  was  made  in  this 
formulation  to  take  into  account  this  functional  relationship. 
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Therefore  the  possibility  exists  that  using  average  unit  costs 
as  an  approximation  could  result  in  a  violation  of  the  budget 
constraint  when  the  true  cost  is  considered.  Another  possible 
result  is  suboptimality  in  the  solution.  Unfortunately,  this 
problem  has  plagued  analysts  for  years,  and  its  solution  was 
beyond  the  scope  of  this  study.  This  problem  should  be 
addressed  in  future  Value  Added  Analysis  applications. 

The  next  set  of  constraints  deals  with  force  structure, 
and  it  specifies  the  range  of  acceptable  values  of  items  onhand 
to  support  established  requirements. 


[9.05] 


®iij  ^  ®ij  ^  ®uij  <  V  i  £  I,  V  j  S  J. 

The  final  set  of  constraints  expresses  both  the  limits 
on  production  capacity  and  the  minimum  allowable  procurement 
quantities  for  each  system. 


[9.06] 


X||j  s  X|j  s  Xyjj ,  V  i  e  I,  V  j  E  J. 

We  observe  that  the  force  structure  and  production 
constraints  can  be  expressed  either  as  explicit  constraints,  or 
as  bounds.  In  the  prototype  version  of  this  optimization,  they 
were  expressed  as  bounds. 

b.  Implementation.  A  C  language  computer  software 
program  was  designed  and  written  to  support  the  Value  Added 
Analysis  effort.  This  program  has  the  following  modules: 

(1)  General  Purpose  Math  Support  Routine  Module. 

This  module  contains  memory  allocation  routines  and  numeric 
conversion  routines  that  are  broadly  applicable  to  all  of  the 
Value  Added  Analysis  support  software.  The  support  routines  are 
based  upon  similar  routines  contained  in  Numerical  Recipes  in  C 

by  Press,  Flannery,  Teukolsky,  and  Vettering.^  This  module  is 
identical  to  that  used  for  TOPSIS  as  described  in  paragraph  9-7. 

(2)  FOMOA  Matrix  Input-OutpUt  Routine  Module. 

This  module  contains  special  purpose  routines  to  read  in  the 
FOMOA  input  data  files  and  allocate  memory  for  the  FOMOA  linear 
programming  formulation.  Of  special  importance  are  the 
procedures  which  determine  the  number  of  rows  and  columns  in 
each  input  matrix.  The  software  is  constructed  so  that  the  size 
of  the  FOMOA  linear  programming  formulation  is  not  predetermined 
and  works  with  large  sizes  limited  only  by  available  memory.  On 


^  Press.  Flannery.  Teukolsky.  and  Vettering,  Numerical  Recipes  in  C.  Oxford 
University  Press.  1988. 
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a  Macintosh  microcomputer  optimization  problems  with  up  to  120 
decision  variables  and  1000  constraints  with  an  average  nonzero 
density  (i.e.,  sparse  matrices)  of  10  percent  have  been  tested. 
Appendix  H  contains  exeunple  inputs  and  outputs  using  the  A3 
illustrative  example. 


9-10 


CAA-SR-91-9 


(3)  FOMOA  Formulation  Input  Module.  This  module 
reads  and  decodes  the  FOMOA  formulation  in  the  algebraic 
language  specified  by  Schwabauer.  The  formulation  language  is  a 
special  purpose  notation  designed  for  the  specification  of 
linear  prograunming  problems.  Its  syntax  is  a  limited  subset  of 
commercially  available  specification  languages  such  as  GAMS  and 
LINGO.  This  module  is  actually  a  series  of  subprograms  which 
lexically  analyze,  parse,  and  interpret  the  FOMOA  language  and 
produce  an  LP  problem  matrix  for  use  by  the  LP  Solver  Module. 

The  FOMOA  language  interpreter  was  written  using  UNIX  parser 
generating  software  ("lex"  and  "yacc")  and  has  a  compact  grammar 
specification  which  conforms  to  standard  UNIX  conventions. 

(4)  Linear  Programming  Solver  Module.  This  module 
contains  the  linear  programming  engine  which  solves  the  problem 
generated  by  the  FOMOA  Formulation  Input  Module.  This  solver 
uses  a  revised  product  form  simplex  algorithm  with  compact 
storage  of  sparse  input  matrices. 

(5)  Main  Program  Module.  This  module  contains  the 
top  level  routine  ("main")  and  calls  all  other  modules.  It 
primarily  serves  to  parse  the  command  line  interface  arguments 
and  open  the  appropriate  input  and  output  files. 

9-5.  SUMMARY.  The  linear  program  formulation  used  for  the 
prototype  VAA  case  study  was  a  simplification  of  the  work 
developed  at  CAA  for  use  in  creating  modernization  plans  for  the 
Office  of  the  Chief  Staff  for  Operations  and  Plans.  We  have 
included  the  more  generalized  form  of  the  formulation  in 
Appendix  H.  Also  included  in  this  appendix  is  a  set  of 
instructions  for  users  of  the  software  developed  for  this 
module.  The  software  currently  is  based  on  Mac  Lingo,  an  off- 
the-shelf  product,  but  could  be  exported  to  other  commercial 
solvers.  This  algebraic  approach  using  commercial  products  we 
feel  provides  a  great  degree  of  flexibility  and  exportability. 
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CHAPTER  10 
SUMMARY  AND  FINDINGS 


10-1.  INTRODUCTION.  The  purpose  of  this  chapter  is  to  provide 
the  summary  and  findings  for  the  Value  Added  Analysis  Study.  Chapter 
10  is  structured  as  follows: 

(1)  Introduction 

(2)  Summary 

(3)  Observations  and  Findings 

(4)  Future  Development 

10-2.  SUMMARY.  The  purpose  of  the  Army  Program  Value  Added 
Analysis  was  to  create  an  analytical  methodology  and  decision 
support  system  for  the  development  of  a  balanced  and  effective 
Army  Program.  We  have  been  successful  in  creating  a  flexible 
and  rational  methodology  to  support  the  POM  process  and  Army 
program  development.  Although  we  were  not  able  to  completely 
automate  the  decision  support  system,  this  study  does  provide  a 
solid  foundation  for  finishing  this  aspect  of  the  study.  The 
team  has  found  that  a  linear  combination  of  value  components 
creating  a  single  measure  of  an  issue's  marginal  value  is 
effective  in  ranking  alternatives  and  conducting  tradeoff 
analyses.  We  were  successful  in  capturing  the  subjective 
elements  used  by  decisionmakers  in  conducting  tradeoffs  between 
alternatives  through  the  use  of  judgment  weights.  This  study 
also  provided  some  important  insights  into  the  POM  decision 
process  which  suggest  areas  to  be  explored  further,  such  as  the 
use  of  aggregated  effectiveness  models,  new  costing  approaches, 
and  issue  clarification  techniques. 

10-3 .  OBSERVATIONS  AND  FINDINGS 

a.  EEA  Observations  and  Findings.  The  observations 
and  findings  contained  in  this  section  of  the  chapter  are  keyed 
to  the  EEA  as  listed  in  the  study  directive. 

(1)  EEA  1:  What  measures  of  effectiveness  (MOE) 

are  appropriate  for  judging  relative  value  among  all 
MDEPs  in  the  program? 

(a)  Two  generalized  categories  of  measures  were 
discovered  to  be  important  in  judging  relative  value.  The  first 
(called  explicit  effectiveness)  consists  of  three  subdivided 
elements  (  combat  effectiveness,  soldier  quality  of  life,  and 
other  Army  goals  and  objectives)  and  includes  those  measures 
which  can  be  directly  quantified.  These  measures  include  such 
MOE  as  fractional  exchange  ratios  (FER),  correlation  of  forces 
and  means  (COFM),  day  care  days,  energy  Btus  saved,  etc.  These 
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MOE  are  selected  for  their  ability  to  appropriately  measure  the 
issue  in  its  own  context.  The  second  set  of  MOE  is  called 
implicit  and  includes  those  measures  which  are  qualitative; 
political  risk,  institutional  stability,  and  importance  of  need 
are  exeimples  of  implicit  measures.  These  MOE  are  selected 
because  they  express  subjective  values  which  are  best  measured 
through  expert  judgment. 

(b)  Viewing  system  effectiveness  as  an  integrated 
piece  of  force  effectiveness,  instead  of  m.easuring  pure  weapon 
system  performance,  yields  a  more  rational  view  of  weapon  system 
contribution  across  systeras  and  mission  areas.  Traditional 
weapon  system-on-weapon  system  performance  used  by  the  combat 
developments  community  does  not  yield  the  insights  required  by 
the  PPBES  progrciming  community.  We  believe  that  the  total  force 
effectiveness  perspective  yields  a  more  useful  measure  for 
programming  decisions.  A  weapon  system’s  "Value  Added"  changes, 
depending  on  the  force  mix  with  which  the  system  interacts,  and 
is  not  normally  captured  using  the  traditional  approach. 

(c)  Implicit  effectiveness  factors  (SIAM  factors) 
beyond  explicit  (pure  combat)  effectiveness  are  a  significant 
component  of  the  DA  decision  process,  and  a  management  survey 
can  yield  important  inputs  into  this  process. 

(d)  No  one  single  measure  of  combat  effectiveness  is 
adequate  to  measure  the  contribution  of  weapon  systems.  More 
work  on  combat  effectiveness  MOE  is  required. 

(e)  Establishing  a  hierarchy  of  MOE  was  found  to  be 
an  effective  way  to  use  results  from  different  models.  The  use 
of  different  levels  of  MOE  in  conjunction  with  sufficiency 
criteria  allows  the  analyst  to  compare  results  from  different 
functional  models  and  use  those  results  as  if  the  models  were 
equivalent . 

(2)  EEA  2:  Should  a  linear  combination  of  value 

components  be  used  to  arrive  at  a  single  measure  of 
value  added?  If  so,  what  system  of  judgment  weights 
should  be  used? 

(a)  A  linear  combination  of  value  components  appears 
to  be  useful  in  creating  a  single  measure  of  an  issue's  marginal 
value . 


(b)  VAA  uses  a  system  of  judgment  weights  which  meas¬ 
ures  the  value  of  an  issue  in  a  dual  context.  The  methodology 
seeks  two  sets  of  estimates.  The  first  set  of  estimates  is 
created  by  surveying  the  Army  leadership  to  determine  both  the 
set  of  judgmental  factors  and  their  weights  in  accordance  with 
the  group's  value  set  and  world  view.  The  second  set  of  esti¬ 
mates  is  created  by  scoring  individual  systems  using  subject 
matter  experts  and  the  judgmental  factors  developed  by  surveying 
the  leadership.  Both  sets  of  surveys  appear  to  yield  consistent 
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and  informative  values.  These  approaches  are  successful  in 
capturing  the  multiple  attributes  used  by  the  decisionmakers  in 
the  PPBES  programming  decision  process. 

(c)  TOPSIS  as  a  method  of  integrating  multiple 
measures  of  effectiveness  into  a  single  measure  of  value  is 
useful  and  effective;  however,  it  may  have  several  analytical 
weaknesses  including  scale  compression,  inadequate  treatment  of 
equivalence  classes  and  sensitivity  to  probabilistic  inputs. 
However,  none  of  i  ,ese  potential  problems  have  been  manifested 
in  this  prototype  proof  of  concept. 

(3)  EEA  3:  What  decision  analysis  framework 

should  be  used? 

(a)  The  VAA  methodology  as  demonstrated  in  Phase  I 
holds  promise  as  the  decision  analysis  framework  that  should  be 
used  for  conducting  Army  program  tradeoff  analyses.  Further  work 
in  the  areas  of  quick  turn-around  combat  modeling, dynamic 
costing, and  data  collection  need  to  be  conducted.  Although  some 
limitations  in  the  methodology  exist,  it  has  been  shown  that 
this  methodology  is  viable  for  the  conduct  of  these  analyses, 
and  will  provide  senior  Army  leaders  with  a  good  and  feasible 
starting  point  for  decision  making,  and  a  way  to  further  analyze 
the  appropriateness  of  various  alternatives. 

(b)  We  believe  that  VAA  is  a  decision  analysis 
framework  that  can  be  used  to  give  decisionmakers  the  capability 
to  group  decisions  by  stakeholder  (interest  group)  and  by  the 
appropriate  context  for  the  issue.  For  example,  the  CINCs  may 
be  one  stakeholder  group  which  would  like  to  look  at  their 
issues  in  a  context  of  combat  operations  based  on  their 
scenarios  and  OPLANs.  Congress  could  be  another  group  whose 
context  is  both  the  Illustrative  Planning  Scenario  (IPS)  and 
constrained  federal  budgets. 

(c)  The  theory  of  cost-benefit  analysis  as  detailed 
in  the  Value  Added  methodology  is  fundamentally  sound.  The 
systematic  specification  of  costs  and  benefits  within  a  consis¬ 
tent  framework  understood  by  both  analyst  and  decisionmaker  is 
an  extremely  positive  feature  of  the  methodology.  The  possi¬ 
bility  of  apparently  anomalous  results  exists,  however.  An 
example  of  such  a  case  that  arose  in  this  study  concerned  TGW. 
This  system  was  determined  to  be  the  number  one  VAA  priority. 
Combat  simulation  results  and  the  explicit  measures  for  the  TGW 
system  were  found  to  be  totally  consistent  with  those  of  TRADOC, 
the  program  manager,  and  RAND  Corporation  analyses.  However,  a 
large  percentage  of  the  senior  Army  leadership  expressed  concern 
about  achieving  the  level  of  results  reported.  As  a  result,  the 
de., ’sion  was  made  to  fund  only  the  RDTE  portion  of  the  program. 
The  positive  aspect  of  this  type  of  occurrence  is  that  a  serious 
issue  was  brought  to  the  attention  of  both  analysts  and  deci¬ 
sionmakers.  Clearly,  additional  work  was  needed  to  show,  one 
way  or  another,  what  could  reasonably  be  expected  with  regard  to 
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the  performance  of  this  system.  Identification  and  illumination 
of  these  issues  is  an  extremely  important  function  of  analysis, 
and  VAA  seems  to  be  very  useful  in  this  area. 

(d)  The  ability  to  modularize  (plug  in,  pull  out) 
components  of  the  Value  Added  methodology  is  critical  to  a 
successful  implementation.  The  analysis  framework  must  be 
standardized,  and  yet  sufficiently  flexible  to  accommodate  a 
variety  of  input  data  and  a  wide  range  of  PPBES  issues.  The 
output  of  each  module  provides  significant  analytical  results  in 
its  own  right.  These  intermediate  results  will  prove  useful  to 
a  variety  of  decisionmakers  at  different  levels. 

(e)  It  is  clear  that  sufficient  information  is  not 
always  present  during  the  construction  of  the  POM  to  provide 
detailed  acquisition  strategies  and  distribution  plans. 
Therefore,  the  VAA  methodology  must  be  capable  of  using 
information  at  various  levels  of  detail  while  still  providing 
meaningful  results. 

(3)  EEA  4:  Does  a  highly  aggregated  combat  model 

provide  useful  insights  for  answering  program-wide 
tradeoff  questions?  If  so,  can  such  a  model  be 
calibrated  to  a  more  detailed  model  if  necessary? 

(a)  A  highly  aggregated  combat  model  can  provide 
useful  insights  for  answering  macro  level  program  tradeoff 
questions.  To  a  more  limited  degree,  such  a  model  is  also 
capable  of  providing  specific  program  guidance.  However, 
decisionmakers  require  sufficient  detail  to  understand  the 
reasons  for  combat  outcomes  to  assist  them  in  specific  program 
decisions.  For  example,  it  was  important  during  the  sponsor 
requested  presentations  to  senior  decisionmakers  that  expla¬ 
nations  be  given  regarding  an  apparent  anomalous  result  in  the 
procurement  of  AH- 6 4  over  the  LHX. 

(b)  In  order  for  aggregated  combat  models  to  be 
useful,  they  must  be  calibrated.  In  order  to  calibrate  them, 
one  must  be  able  to  establish  some  landmark  values  or  calibra¬ 
tion  points  using  data  or  output  from  accepted  and  "known" 
sources.  These  points  do  not  necessarily  need  to  be  developed 
from  other  models  of  higher  resolution.  Calibration  can  be  done 
using  other  models,  empirical  data,  or  historical  data. 
Calibrating  one  model  with  results  from  another  has  been  a 
commonly  used  procedure  in  hierarchical  analysis. 

(4)  EEA  5:  What  Blueprint  (of  the 

battlefield)  options  provide  the  most  useful 
framework  for  the  analysis  of  change  to  the  Army's 
materiel,  force  structure,  etc?  TRADOC's  Blueprint  of 
the  Battlefield,  which  this  EAA  was  directed  towards, 
appears  not  to  be  useful  within  the  context  of  VAA.  The 
blueprint  concept  is  oriented  by  its  very  nature  toward  a 
requirements  perspective  and  does  not  necessary  address  the 
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issues  of  affordability  or  duplication  of  capability.  The 
AHP  process  appears  to  hold  the  greatest  possibilities  in 
this  regard. 


b.  Other  Observations  and  Findings.  The  observations 
and  findings  contained  in  this  paragraph  of  the  chapter  are  not 
necessarily  keyed  to  the  original  questions  proposed  in  the 
study  directive  but  are  insights  obtained  as  a  result  of  the 
work  completed  in  this  study. 

(1)  The  Value  Added  methodology  is  data  intensive. 
Accuracy  and  timeliness  of  input  data,  especially  production 
data,  is  of  crucial  importance  in  making  the  value  added  process 
operational . 

(2)  The  initial  goal  of  value  added  was  to  be  quick 
turnaround  for  all  components.  What  was  found  was  that  an 
extensive  front-end  process  (especially  combat  modeling  and 
costing)  was  required  which  consumed  large  amounts  of  time. 

Once  this  front  end  work  was  completed,  then  quick-response 
excursions  were  possible. 

(3)  The  Value  Added  methodology  will  "buy"  the  most  cost 
effective  systems  subject  to  constraints. 

(4)  The  study  developed  a  modular  "living"  methodology 
with  extreme  flexibility  that  can  be  used  with  a  variety  of 
techniques  and  models. 

(5)  The  methodology  provides  a  standardized  quick 
reaction  approach  that  uses  ORSA  techniques.  It  evaluates 
program  tradeoff  issues  by  functional  area  experts  and  deci¬ 
sionmakers.  ORSA  expertise  is  required  during  the  preparatory 
analysis,  consisting  of  the  combat  modeling,  costing,  and 
surveying.  The  functional  analysis  which  would  take  place 
during  the  POM  building  and  defense,  consisting  of  the  actual 
tradeoffs,  do  not  require  extensive  ORSA  expertise  when  using 
the  VAA  methodology. 

(6)  Issue  clarification  is  difficult,  and  it  appears 
that  senior  decisionmakers  have  difficulty  focusing  issues. 
Therefore,  issue  clarification  is  key  to  the  success  of  the 
follow-on  modules  and  analysis. 

(7)  The  technique  used  to  survey  the  Army  leadership 
allows  decisionmaking  attitudes  and  behaviors  to  be  accurately 
modeled.  Administration  of  the  management  survey  under  tightly 
structured  and  controlled  conditions  yielded  statistically 
reliable  results. 
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10-4.  VAA  FUTURE  DEVELOPMENT 

a.  Methodology.  The  study  team  believes  the  following 
methodological  improvements  or  research  should  be  conducted  to 
enhance  the  VAA  capability 

(1)  The  issue  clarification  module  requires  better 
definition  to  include  appropriate  models  and  algorithms. 

(2)  Research  is  needed  into  the  use  of  response  curves 
or  some  other  emulation  method  for  developing  a  quick  turnaround 
combat  modeling  capability. 

(3)  The  analytical  hierarchy  portion  of  the  implicit 
effectiveness  module  must  be  completed.  It  must  include  the 
level  3  elements  of  the  soldier  quality  of  life  and  other  Army 
goals  and  objectives. 

(4)  The  ability  to  provide  a  "supervalue"  for  a  par¬ 
ticular  SIAM  factor  needs  to  be  incorporated  into  the  method¬ 
ology.  For  example,  a  particular  issue  or  piece  of  equipment 
may  have  high  operational  risk  associated  with  it.  This  risk 
may  be  so  important  that  it  overshadows  all  other  factors,  to 
include  the  explicit  measures  of  value.  The  methodology  should 
identify  these  types  of  issues  during  the  process. 

(5)  The  methodology  should  be  capable  of  handling 
uncertainty  associated  with  the  scores  SMEs  give  to  an  issue  or 
piece  of  equipment.  The  process  may  include  some  kind  of 
parametric  sensitivity  mechanism  to  accommodate  the  imprecision 
of  the  values  provided. 

(6)  Research  into  the  potential  TOPSIS  weaknesses  of 
scale  compression,  inadequate  treatment  of  equivalence  classes 
and  sensitivity  to  probabilistic  inputs  must  be  performed. 

(7)  A  more  complete  and  detailed  formulation  of  the 
optimization  LP  must  be  determined  in  order  to  better  address 
decisionmakers'  concerns. The  LP  should  include  dynamic  costing 
for  procurement  costs. 

(8)  An  effective  methodology  for  presenting  VAA  results 
must  be  developed  to  more  clearly  convey  information  to  the 
decisionmakers . 

b.  Implementation.  The  study  team  believes  the 
following  implementation  questions  should  be  addressed  in 
follow-on  work. 

(1)  A  VAA  desk  top  for  the  METAPHOR  architecture  to 
include  associated  icons  and  capsules  must  be  developed  and 
implemented. 
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(2)  Software  to  implement  the  issue  clarification  models 
and  algorithms  must  be  developed.  ARIM,  as  part  of  this  effort 
for  both  issue  clarification  and  costing,  must  be  fully 
implemented. 

(3)  The  CQM  and  LCCM  models  in  the  METAPHOR  environment 
must  be  implemented. 

(4)  A  family  of  combat  effectiveness  models  to  include 
an  additional  force  level  model  and  appropriate  functional  area 
models  must  be  developed. 

(5)  The  implementation  of  expert  systems  to  assist  in 
capturing  the  experience  of  SMEs  and  decisionmakers  should  be 
explored. 

(6)  An  automated  or  networked  collection  system  may  be 
helpful  in  collecting  the  data  needed  to  build  weights  and 
scores . 


c.  Management.  The  study  team  believes  the  following 
management  issues  should  be  considered  in  the  future. 

(1)  The  survey  technique  needs  to  be  institutionalized 
within  the  Department  of  the  Army  to  include  groups  to  be 
surveyed,  timing  in  relation  to  the  PPBES  cycle,  documentation 
of  the  survey  process,  and  development  of  criteria  for  selecting 
survey  teams. 

(2)  A  default  standardized  Value  Added  Analysis  approach 
should  be  institutionalized. 

(3)  The  scope  of  value  added  analysis  needs  to  be 
expanded  beyond  the  RDA  appropriations  since  these  contain  only 
about  22  percent  of  the  Army  program  dollars. 
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APPENDIX  8 
STUDY  DIRECTIVE 
department  of  the  army 
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ML'IORANDUM  FOR  DIRECTOR.  CONCEPTS  ANALYSIS  AGENCY 

DEPUTY  CHIEF  OF  STAFF  FOR  CO.MBAT  DEVELOP.NENTS ,  TRADOC 

SUBJECT;  Aray  Prograa  Value  Added  Analysis  (Value  Added  Analysis  90-97) 


1.  PURPOSE  OF  STUDY  DIRECTIVE:  This  Bemorandun  Casks; 

a.  The  U.S.  Army  Concepts  Analysis  Agency  (CAA)  in  coordination  vith  the 
PaED  Modeling  Cell  and  the  ODCSOPS  Technical  Advisor  to  develop  a  program 
trade-off  analysis  methodology  chat  fits  within  the  Value  Added  concept  as 
developed'by  PAED  for  building  the  POM  (see  enclosure  1). 

b.  US.  Army  Training  and  Doctrine  Command  (TRADOC),  in  particular  TRADOC 
Analysis  Command  (TRAC),  to  provide  Corps  level  force  on  force  analysis 
support.  Efforts  in  support  of  the  Array  Requirements  and  Capabilities 
Studies  (ARCPS)  will  be  Intorporaced  wherever  possible  to  cake  advantage  of 
useful  methods,  data,  and  methodologies;  and  provide  consistency  between 
'ARCPS  and  the  Value  Added  Analysis. 

2.  STUDY  TITLE:  Army  Prograa  Value  Added  Analysis  (Short  title:  Value  Added 
Analysis  90-97) 

3.  BACKGROUND ;  HQDA  needs  c.ialysis  to  support  the  development  of  a  balanced 
and  effective  Array  Program  within  Dapartmenc  of  Defense  resource  guidance. 
Traditionally  this  has  been  accomplished  during  the  POM  building  process  with 
functional  area  panels.  Tie  processes  that  have  evolved  do  not  Include 
adequate  means  of  integrating  and  balancing  the  functional  prograrcs  into  the 
total  Army  Prograa. 

a.  Early  in  the  process  TOA  is  provided  to  each  panel  and,  within  the 
panel,  a  prioritization  is  conducted.  The  methods  used  vary  from  panel  to 
panel  and  subjective  Judgement  is  often  used. 

b.  Virtually  all  analysis  currently  performed  in  program  evaluation 
focuses  on  defining  individual  Management  Decision  Package  (MDEP)  Issues. 

When  MDEP  analysis  is  conducted  the  senior  Array  leadership  lacks  the  analysis 
necessary  to  help  identify  the  marginal  value  ol  resources  within  a  .MDEP  or 
across  MDEPs. 

c.  Analysis  of  the  total  Army  Progra.m  requires  an  understanding  of  how 
individual  KDEP  contribute  to  the  Army  mission  and  strategy  so  as  to 
deter.mlne  which  MDEPs  and  resource  levels  have  the  greatest  return  on 
Investment  and  to  assess  ai fordablllty .  One  approach  to  this  problem  is  to 
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estimate  the  value  added  by  Individual  MDEPs ,  or  groups  of  MDEPs  (expressed 
as  program  enhancement  packages)  to  the  Army  as  measured  by  their 
contribution  to  the  warfighclng  capability  of  the  Army. 

d.The  Value  Added  Analysis  concept  developed  by  PAED  uses  a  hierarchial 
assessment  framework  for  assessing  how  resource  changes  relate  to  battlefield 
capability.  This  assessment  framework  is  used  to  portray  existing  capability 
and  measures  shortfalls  against  a  postulated  objective  requirement.  The 
contribution  of  program  alternatives  (enhancement  packages)  will  be  evaluated 
within  Che  hierarchial  fra.mework.  The  methodology  is  based  on  using  combat 
simulations  and  models  to  identify  how  alternatives  .impact  relevant  .“.OEs. 
These  relationships  in  conjunction  with  the  funding  resources,  will  form  the 
input  values  which  will  be  used  in  a  decision  analysis  framework. 

4.  STUDY  SPONSORS: 

a.  The  Director  Program  Analysis  and  Evaluation  (D?AE) ,  Office  of  the 
Chief  of  Staff  Army,  is  a  co-sponsor.  DPAE  is  responsible  for  the  Army 
Program  and  for  building  the  Program  Objective  Memorandum  (PO.M). 

b.  The  Technical  Advisor  to  the  Deputy  Chief  of  Staff  for  Operations  and 
Plana  is  a  co-sponsor.  The  Technical  Advisor  is  responsible  for  providing 
advice  and  assistance  on  analytical  issues  to  the  OCSOFS. 

c.  The  Study  Sponsor's  Representative  is  KAJ  Robert  R.  Koury,  PAED 
(OACS-DPZ). 

5.  STUDY  AGENCIES:  CAA  is  the  lead  study  agency. 

a.  CAA  will  Identify,  and  where  necessary  develop,  models  and  quantitative 
techniques  to  assist  in  th»  refinement  and  implementation  of  analytic  methods 
for  estimating  the  margina.l  value  to  the  Army  of  competing  MDEPs. 

b.  TRAC  will  assist  CAA  as  required. 

c.  The  U.S.  Army  Materiel  Command  (AMC) ,  the  US  Army  Cost  and  Economic 
Analysis  Center  (CEAC) ,  and  ocher  agencies  will  assist  as  required. 

6.  aajE£iiyE,£i. 

a.  Formulate  an  analytic  method  for  estimating  the  marginal  value  of 
competing  MDEPs  to  the  Army.  The  methodology  will  provide  a  common 
understandable  basis  for  the  analysis  of  affordability  issues. 

b.  Identify  or  develop  prototype  models  that  support  the  Value  Added 
Analysis  Methodology  and  provide  management  tools  for  the  analytic  method 
developed  in  a,  above. 
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SUBJECT:  Army  Program  Value  Added  Analysis  (Value  Added  Analysis  90*97) 


c.  Escablish  within  PaED  an  in-house  capability  to  conduct  program  issues 
trade -ofts. 

d.  Conduct  a  proof  of  concept  analysis  during  the  building  of  the  92-97 

POM. 

7.  TERMS  OF  REFERENCE : 

a.  Use  Amended  Budget  r(90/91  and  Program  Force  (FY94)  As  Baseline 

b.  Scenario  Conditions:  NATO  Central  Europe,  IPS  Scenario 

TRADOC  approved  scenario(s) 

Scenario  variations  necessary  for  sensitivity 
analysis. 

c.  Conflict  Duration:  TBD 

d.  Conflict  Type:  Conventional 

a.  Measures  of  Effectiveness:  (For  Example) 

*  Independent  Measures  (Note  1) 

Blue  Force:  Surviving 
Correlation  of  Forces  and  Means 

*  Dependent  Measures  (Note  2) 

Air  Defense  -  Time  Air  Space  Denied 
CSS  -  Short  Tons/  Day 

Close  Combat  Force  Exchange  Ratio 

Note  1:  Establish  an  overall  force  effectiveness. 

Note  2:  Establish  effectiveness  values  for  specific  battlefield 
functional  areas. 


8.  limits ; 

a.  Time  for  developing  the  prototype  analysis  will  not  exceed  1  August 
1989. 

b.  Since  this  initial  effort  will  be  a  proof  of  concept  validation,  the 
analysis  will  exam  only  the  RDA  appropriations.  However,  the  final 
methodology  must  eventually  be  capable  of  examining  trade-offs  across  the 
full  range  of  program  issues.  The  analysis  will  expand  to  the  ocher 
appropriations  as  the  methodology  matures. 
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SUBJECT:  Array  Ptograra  Value  Added  Analysis  (Value  Added  Analysis  90-97) 


c.  Fresldenc  Reagan's  Budget  poslclon  will  be  used  for  both  PROBE  and 
ROAISA  dacabases. 

d.  Software  must  operate  in  an  IBM  PC  compatible  environment. 


9,  ASSUMPTIONS  AND  THEIR  SIGNIFICANCE:  The  study's  major  assumptions  are 
listed,  below.  Scenario-based  assumptions  will  be  described  in  the  final  study 
report  annex  dealing  with  the  warfight  scenarios; 

a.  Any  set  of  value  added  models  to  be  used  In  the  POM  building  process 
must  be  quick  turn-around  and  IB.M  PC  compatible  based.  Because  the  Value 
Added  Analysis  raodels  oust  execute  in  a  quick  turn-around  environment,  the 
inherent  combat  models  must  process  aggregated  data  and  may  need  to  be 
calibrated  to  higher  fidelity  raodels  as  a  means  of  increasing  their 
credibility. 

SIGNIFICANCE;  This  assumption  drives  us  to  a  low-resolution  model 
which  captures  the  effects  and  Interactions  of  the  warfight;  NOT  necessarily 
the  process  of  the  warfight  Itself. 

b.  The  Program  Value  Added  Analysis  oust  accommodate  the  manner  in  which 
the  senior  Amy  leadership  makes  decisions. 

SIGNIFICANCE;  The  decision  framework  and  models  developed  must  be 
acceptable  to  the  senior  Aimy  leadership. 

c.  The  Program  Value  Added  Analysis  must  allow  for  prioritizing  between 
dissimilar  MDEPs,  i.e.,  Weapon  system  procurement,  logistical  improvements, 
force  structure  changes,  etc. 

SIGNIFICANCE;  In  order  to  provide  a  means  to  trade-off  between 
different  MDEPs,  the  analysis  must  be  capable  of  relating  the  MDEPs 
contribution  to  the  Army  warfight  capability. 

d.  Input  data  will  come  from  a  wide  variety  of  sources  and  may  prove  to 
be  difficult  to  obtain  at  times. 

SIGNIFICANCE:  This  lack  of  data  will  be  handled,  most  likely,  by 
parametric  analysis  for  those  data  elements  which  cannot  be  obtained. 


10.  ESSENTIAL  ELEMENTl.  OF  THE  ANALYSIS; 

a.  What  dependent  measures  of  effectiveness  (HOEs)  are  the  most 
appropriate  means  of  judging  relative  value  added? 
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SUBJECT:  Army  Program  Value  Added  Analysis  (Value  Added  Analysis  90-97) 

b.  What  system  of  Judgement  weights  should  be  used? 

c.  What  decision  analysis  framework  should  be  used? 

d.  Does  a  highly  aggregated  macro  level  analytical  methodology  provide 
useful  Insights  for  answering  total  Army  program  trade-off  questions? 

e.  What  Blueprint  options  provide  the  most  useful  framework  for  the 
analysis  of  change  to  the  Army's  materiel,  force  structure,  etc.? 


11.  REgfOJiSmUTISSl 

a.  D.PAE  will: 

(1)  Provide  the  study  sponsor  representative  and  study  guidance. 

(2)  Provide  a  study  team  member. 

b.  Technical  Advisor  C>>  the  DCSOPS  will: 

(1)  Provide  study  guidance  to  the  study  sponsor’s  representative. 

(2)  Provide  a  study  team  member. 

<3)  Coordinate  the  input  of  ARCPS  data  Co  the  study. 

c.  CAA  will: 

(1)  Designate  a  study  director  and  provide  the  study  team. 

(2)  Coordlnate/communlcate  with  the  PAED,  ODCSOPS,  TRAC,  CEAC,  and  AMC 
for  data  and  information  to  accomplish  the  study. 

(3)  Provide  periodic  in-process  reviews  (IPRs)  as  requested  by  the 

study  sponsors  and  provide  the  study  products  as  listed  in  paragraph  12  of 
this  tasker.  ^ 

(4)  Assist  PAED  in  establishing  an  in-house  capability  for  linking  the 
prototype  models  and  analytical  tools  into  a  coherent  analytical  approach. 

(5)  Assist  PAED  in  conducting  a  proof  of  concept  analysis  using  the 
prototype  models  on  specific  program  issues  which  will  be  identified  prior  to 
the  building  of  the  92-97  POM  (1  July  89). 
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SUBJECT:  Army  Program  Value  Added  Analysis  (Value  Added  Analysis  90-97) 


12.  STUDY  PRODUCTS :  CAA  will  produce,  in  coordination  with  TRAC  and  Che  PaE 
Modeling  Ceil: 

a.  A  final  study  report  which  will  thoroughly  documents  the  Value 
Added  Methodology. 

b.  A  working  Value  Added  Methodology  to  include  the  underlying  analytic 
concepts,  software  and  hardware  linkages. 

c.  Briefings  as  required. 

13.  ADMINISTRATIVE:  . 

a.  Support. 

(1)  Funds  for  travel/per  diem  will  be  provided  by  the  parent 
organization  of  each  study  participant. 

(2)  Clerical  support  will  be  provided  by  CAA  and/or  TRAC  as  required 

(3)  ADPE  support  will  be  provided  by  CAA  and  by  TRAC  as  required. 

b.  Milestone  Schedule.  The  proposed  schedule  is  at  enclosure  2 

c.  Control  Procedure.  The  study  sponsor's  representative  will  provide 
guidance  within  the  HQDA  staff  and  between  Che  aRSTAFF  and  the  study 
agencies. 

d.  Coordination.  This  directive  has  been  coordinated  with  CAA  and  TRAC. 


2  Enel  william  H.  RENO 

Major  General.  CS 
Director,  Program  Analysis 
'  and  Evaluation. 


y^JOm  A.  RIENTE 
A'  Technical  Advisor  to  the 
Deputy  Chief  of  Staff  for 
Operations  and  Flans 
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APPENDIX  D 

VAA  MODULE  MATRIX  (  PROCESSESS,  TECHNIQUES  AND  PRODUCTS) 

G-1.  INTRODUCTION.  The  VAA  module  matrix  was  developed  to 
provide,  in  a  single  document,  an  outline  of  the  methodology  and 
the  processes  associated  with  the  VAA  concept. The  matrix  is 
divided  into  two  parts.  Each  section  represents  a  module  of  the 
methodology  and  is  labeled  with  the  module  name.  The  first  part 
outlines  the  processess,  techniques  and  products  associated  with 
each  module.  The  second  part  is  organized  to  relate  the  input 
data,  sources  of  data  and  assumptions  back  to  each  process  and 
module. 

G-2.  MATRIX  ORGANIZATION.  This  section  will  briefly  outline 
each  of  the  matrices; 

a.  Annex  1  Matrix  1.  There  are  eight  sections  to  this 
matrix  covering  seven  of  the  eight  modules.  No  discussion  of  the 
results  and  display  module  is  provided  because  it  is  yet  to  be 
determined  and  will  be  defined  as  part  of  VAA  Phase  II. 

Processes  are  located  in  the  second  column  of  the  matrix  and  are 
linked  to  subsections  of  the  modules.  The  intent  of  this  matrix 
was  to  show  the  relationship  within  and  between  modules  and  how 
processes  are  related  to  analytical  techniques. 

b.  Annex  2  Matrix  2.  There  are  also  eight  sections  to 
matrix  2  covering  seven  of  the  eight  modules .  Processes  are 
located  in  the  second  column  of  the  matrix  in  a  short  hand  style 
and  are  linked  to  subsections  of  the  modules.  The  intent  of  this 
matrix  was  to  show  the  relationship  of  input  data  to  the  modules 
and  processes  of  the  VAA  Methodology.. 
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APPENDIX  P 
COST  LEARNING  CURVES 
F-1.  INTRODUCTION 

a.  Experience  curves  have  been  a  tool  historically 
employed  by  government  and  industry  to  forecast  production 
costs.  Their  origins  lie  chiefly  in  the  aircraft  industry  of 
the  1930s  and  World  War  II.  Relying  on  data  from  previous 
production  experience,  these  curves  map  out  a  relationship 
between  the  costs  of  producing  a  particular  good  and  the 
quantity  of  that  good  made.  The  costs  can  be  measured  in 
various  forms,  such  as  labor  input  per  unit  or  dollars  per 
unit.  Although  they  were  originally  utilized  in  the 
production  of  aircraft,  they  are  easily  applied  to  other 
products  (such  as  tanks,  trucks,  etc.). 

b.  When  speaking  of  experience  curves,  one  is  referring 
to  the  general  case.  There  are  many  factors  involved  in  the 
production  of  a  good.  The  two  most  common  are  capital  and 
labor.  Each  factor  of  production  has  its  own  underlying 
experience  curve.  For  example,  the  curve  for  labor  is  often 
referred  to  as  the  "learning  curve."  Thus,  the  term 
"experience"  can  be  used  with  labor,  machinery,  management, 
etc.  It  is  a  general  concept  that  applies  to  all  the  factors 
of  production. 

F-2.  EXPERIENCE  CURVE  THEORY 

a.  Experience  curve  theory  simply  states  that  as  produc¬ 
tion  of  a  particular  good  doubles,  the  production  cost 
decreases  at  some  constant  rate.l  This  is  due  to  the 
"experience"  aspect  of  performing  tasks  repetitively.  As  an 
example,  consider  the  case  of  labor.  As  workers  perform 
tasks  again  and  again,  they  become  more  efficient  in 
executing  a  given  task.  Since  worker  efficiency  rises,  less 
labor  input  is  required  in  the  production  of  additional 
units.  This  reduces  production  costs. 

b.  In  general,  experience  curve  theory  appears  in  two 
forms.  One  refers  to  the  incremental  unit  costs  of  produc¬ 
tion.  This  form  states  that  as  production  doubles,  the  per 
unit  cost  of  a  good  at  a  given  quantity  will  decrease  at  a 
constant  rate.  The  other  form  is  similar  and  deals  with  the 
average  costs  of  production.  Here,  the  theory  states  that  as 
production  doubles,  the  average  costs  of  production  up  to  a 
given  quantity  will  decrease  at  a  constant  rate.  For 
simplicity,  the  following  discussion  will  refer  to  experience 


*Cost  Estimating  for  Engineers  Correspondence  Text  (CEE), 
ALM-63-3126-LC(C).  Page.  201. 
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curves  on  the  incremental  cost  basis.  Any  conclusions  can  be 
easily  extended  to  the  average  cost  basis. 

c.  There  is  a  whole  host  of  factors  that  contribute  to 
the  decrease  in  production  costs  as  output  rises. 2  Some  of 
these  include  learning  on  the  behalf  of  management,  machinery 
improvements,  technological  change,  etc.  All  of  these 
factors  will  have  a  substantial  impact  on  production  costs. 
Despite  this,  it  is  viewed  by  some  that  worker  learning 
through  repetition  is  the  dominant  factor  in  experience  curve 
theory. 3  This  notion  of  the  dominant  role  of  labor  is  not 
universally  accepted. ^ 

F-3.  BASE  EXPERIENCE  CURVE 

a.  The  experience  curve  is  traditionally  represented  by 
the  following  function: 

1)  y-zur* 

where: 

y  =  unit  cost 

A  =  cost  of  the  first  unit 

X  =  the  unit  number 

b  =  experience  curve  slope  coefficient. 

The  slope  coefficient  (b)  is  usually  constrained  to  lie 
between  zero  and  negative  one.  That  is, 

2)  -Is  b  sO. 

The  slope  coefficient  (b)  can  be  viewed  as  the  marginal 
change  in  the  unit  cost  ( Y)  that  would  occur  if  one  more  unit 
of  output  (X)  were  produced.  Given  (2),  it  can  easily  be 
verified  from  (1)  that  as  (b)  approaches  zero,  the  experience 
curve  effect  diminishes.  In  fact,  should  (b)  take  the  value 
zero,  then  unit  cost  (  y)  would  be  equal  to  the  first  unit 
cost  (A). 5  This  implies  that  increases  in  the  number  of 
units  produced  has  no  impact  upon  unit  cost.  That  is,  there 
is  no  experience  effect.  This  would  be  termed  a  100  percent 
experience  curve. 


-  See  CEE  ALM-63-3126-LC(C),  Page  201,  and  Smith,  Jason.  1989.  Learning 
Curve  for  Cost  Control.  Institute  of  Industrial  Engineers.  Pages  14  and  15. 

^This  will  be  more  true  for  those  industries  that  are  labor-intensive. 

“^See  Smith,  Jason.  1989.  Learning  Curve  for  Cost  Control.  Institute  of  Industrial 
Engineers.  Pages  13-17,  20. 

^Let  (b)  equal  zero.  This  implies  will  always  equal  one.  From  (1),  this 

reveals:  Y=A.  In  a  graph  with  the  horizontal  axis  the  number  of  units  and  the 

vertical  axis  being  unit  cost,  there  would  be  a  horizontal  line  at  Y=A. 
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b.  The  exa<^.t  opposite  occurs  as  (b)  approaches  (-1). 

Here,  the  expedience  effect  becomes  large.  Should  (b)  equal 
(-1),  then  a  doubling  of  the  nxunber  of  units  produced  would 
lead  to  a  50  percent  reduction  in  unit  cost.  This  outcome 
would  be  called  a  50  percent  experience  curve. 

F-4.  ESTIMATING  THE  BASE  EXPERIENCE  CURVE 

a.  The  most  precise  method  of  estimating  the  experience 
curve  is  through  the  use  of  ordinary  least  squares  regression 
(OLSQ) .  To  do  this,  equation  (1)  must  first  be  put  in  linear 
form.  From  above, 

l.a)  Y-AX'”. 

Note  that  the  learning  curve  exponent  (b)  is  negative.  This 
is  due  to  the  constraint  placed  by  (2).  Taking  the  logarithm 
of  (l.a)  yields, 

3)  ln(y)-In(A)-feln(;iO. 


Since  equation  (3)  is  now  log-linear,  it  can  be  estimated  by 
OLSQ.  The  resultant  regression  coefficients  will  be  In (A) 
and  (b).  To  determine  the  first  unit  cost  (A),  the  anti-log 
of  ln(A)  must  be  taken.  The  experience  curve  slope  (b)  is 
found  directly  from  its  estimated  coefficient. 

b.  The  experience  curve  percentage  can  be  determined  in 
the  following  manner.®  Allow  {%)  to  represent  the  fraction 
to  which  unit  cost  decreases  as  production  doubles.  Then, 


4)  %- 


2k 


where : 


Y  =  unit  cost 

Yx  =  the  unit  cost  of  the  xth  unit 
Yzx  =  the  unit  cost  of  twice  the  xth  unit. 

Substitution  from  (1)  above  gives 


>2,  _Ai2x)^  fa 
Kt  AX^ 

This  can  be  rewritten  as 


^RAND  Corporation,  The  Learning  Curve,  RM-6103-SA.  Pages  96-97. 
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From  (6),  the  experience  curve  percentage  can  be  determined. 


c.  Consider  the  following  example.  Unit  cost  and  quantity 
data  for  a  certain  weapons  system  is  presented  in  Table  F-1. 
These  variables  will  be  represented  by  ( F)  and  (X),  respec¬ 
tively.  Also  listed  in  the  table  are  the  logarithm  values  of 
each  variable.  To  estimate  the  experience  curve  for  this 
system,  equation  (3)  must  first  be  estimated.  This  is  done 
by  utilizing  OLSQ  regression.  From  the  data  in  Table  F-1, 
the  following  OLSQ  results  were  found: 

3.  a)  ln(y)- 2.002  -  0.1181n(X). 


_ Table  F-1. _ 

Cost  &  Quantity  Data  for  a 
Given  Weapon  System 

(cost  data  in  millions  of  dollars) 


ln(y) 

ln(X) 

1996 

4.995 

1.608 

29.15 

3.372 

1997 

3.869 

1.353 

234.23 

5.456 

1998 

3.451 

1.239 

657.84 

6.489 

1999 

3.225 

1.171 

1168.18 

7.063 

2000 

3.072 

1.122 

1672.19 

7.422 

2001 

2.980 

1.091 

2174.34 

7.684 

2002 

2.923 

i.073 

2675.68 

7.892 

2003 

2,892 

1.062 

3137.04 

8.051 

The  "hat"  above  the  unit  cost  variable  {Y)  denotes  it  as  the 
predicted,  or  fitted  value  of  (Y) . 

d.  From  (3. a)  the  slope  coefficient  (b)  can  be  taken 
directly.  Here,  (b)  is  equal  to  (-0.118).  The  first  unit 
cost  value  (A)  is  not  as  simply  determined.  From  (3)  above, 
it  can  be  seen  that  the  OLSQ  regression  of  (3. a)  gives  the 
log  value  of  (A) .  Since  we  are  not  interested  in  Lhe 
logarithmic  value  of  (A),  but  (A)  itself,  the  anti-log  of  (A) 
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must  be  taken. ^  Doing  so  reveals  that  (A)  is  equal  to 
7.406.8 


«*.  The  experience  curve  percentage  can  be  found  by 
applying  equation  (6): 

6. a)  ln(%)  =  b  ln(2) 

=  (-0.118) (0.693) 
ln(%)  *  -0.081774. 


Note  that  we  are  interested  in  (%)/  not  its  logarithm.  Thus, 
the  anti- log  of  [ln(%)]  must  be  found.  Doing  so  reveals, 

%  =  0.92  or  92  percent. 

Below  is  a  summary  of  the  estimated  experience  curve  from  the 
data  in  table  F-1: 

Estimated  Equation;  y-7.406A''®^^® 

Slope  of  cost/quantity  line  =  0.92 

F-5.  COST/QUANTITY/RATE  METHOD 

a.  It  is  widely  recognized  that  the  rate  of  production 
plays  an  important  role  in  unit  production  costs.  One  method 
of  accounting  for  this  effect  has  been  proposed  by  Bemis.^ 

He  has  altered  the  traditional  experience  curve  in  two  ways. 
First,  the  functional  form  of  the  experience  curve  has  been 
changed  to: 


7) 

where 


Y  =  average  unit  cost 
Q  =  cumulative  lot  mid-point 
R  =  production  rate 
b  =  cost/quantity  slope  coefficient 
c  =  cost/rate  slope  coefficient. 


^This  can  easily  be  done  on  a  hand-held  calculator.  Simply  take  the  log  value 
in  question,  and  press  the  (inverse)  and  (In)  keys  (in  that  order). 

^  The  regression  found  ln(A)=2.002  (the  constant  term).  The  anti-log  of  this 
value  is  equal  to  7.406.  See  footnote  (7). 

^  Bemis,  John  C.  The  Cost  Implications  of  Production  Rate.  Concepts,  Spring 
1981,  Volume  4,  Number  2.  Pgs.  84-94, 
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In  this  revised  form,  unit  cost  is  a  function  of  both  the 
cumulative  lot  mid-point  ( a  measure  relating  unit  cost  and 
quantity  produced)  and  the  production  rate. 

b.  The  variable  cumulative  lot  mid-point  is  the  second 
alteration.  It  serves  as  a  measure  relating  unit  cost  to  the 
total  quantity  produced.  This  is  necessary  because  manu¬ 
facturers  generally  do  not  keep  cost  data  on  individual  units 
of  production.  Instead,  they  maintain  cost  data  based  on 
"lots"  of  output.  The  number  of  units  in  a  given  lot  will 
also  be  known.  To  determine  the  experience  curve,  the  unit 
number  and  its  associated  coat  is  needed.  The  cumulative  lot 
mid-point  is  a  method  to  adjust  the  lot  data  so  the 
experience  curve  can  be  estimated. 

c.  The  mid-point  of  any  given  lot  can  be  estimated  by  the 
algebraic  lot  mid-point  {ALM) .  The  algebraic  lot  mid-point 

is  the  unit  in  a  given  lot  whose  cost  is  equal  to  the  average 
cost  of  all  the  units  in  the  lot.  It  can  be  found  using  the 
following  estimator: 


8) 

4 

where 

ALM  -  approximate  algebraic  lot  mid-point 
F  *  first  unit  in  the  lot 
L  *  last  unit  in  the  lot. 

Note  that  the  variable  (Q)  appears  in  equation  (7).  This 
signifies  that  the  production  lot  data  should  be  converted  as 
in  ( 8 ) .  Equation  ( 8 )  must  be  applied  in  the  appropriate 
manner . 


d.  Calculating  the  cumulative  lot  mid-point  {Q)  can  most 
easily  be  shown  by  an  example.  Consider  the  average  uniu 
cost  { y)  and  production  rate  (J?)  data  in  table  F-2 .  For 
FY96,  (Q)  is  found  in  the  following  manner: 

^:FY96  "  ~ 


U96>2V(1)(96) 

4 


/116.6N, 

“I  4  I 


Qfy96  *  29.15. 
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Table  F-2. 

Cost,  Quantity,  and  Rate  Data  for  a 

Given  Weapon  System 

(cost  data  in  millions  of  dollars) 

Production 

Lot  mif- 

Fiscal  year 

Unit  cost(n 

rate  (R) 

1996 

4.995 

96 

29.15 

1997 

3.869 

335 

234.23 

1998 

3.451 

500 

657.84 

1999 

3.225 

500 

1168.18 

2000 

3.072 

500 

1672.19 

2001 

2.980 

500 

2174.34 

2002 

2.923 

500 

2675.68 

2003 

2.892 

418 

3137.04 

For  FY  97,  the  cumulative  lot  mid-point  is; 


Qfyvj 


F  +  L->-  i4fL 
4 


97-t-43U2V(97)(431) 

4 


“  234.23. 


The  remaining  entries  in  Table  F-2  can  be  found  in  a  like 
fashion. 

e.  Estimation  of  the  cost/quantity/rate  equation  is 
similar  to  the  base  experience  curve.  OLSQ  regression  is 
performed  on  the  log  transformed  variables  of  (7).  The 
dependent  variable  is  average  unit  cost  ( Y) ,  and  the 
explanatory  variables  are  the  cumulative  lot  mid-point  (Q) 
and  production  rate  (R).  The  cost/quantity/rate  equation  to 
be  estimated  is, 

9)  ln(r)-ln(A)-feln(C)-cln(R).  [D.12] 

From  the  regression  results  of  (9)  the  slope  coefficients  can 
be  taken  directly. 

f.  The  data  in  Table  F-2  can  serve  as  an  excunple  of  the 
cost /quantity /rate  method.  Estimating  (9)  with  this  data  by 
OLSQ  reveals: 

10  )  ln(Y)  -  2.04  -  0.1141n(Q)  -  0.0121n(R). 

Following  the  techniques  previously  presented,  the  follow¬ 
ing  was  determined: 
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Estimated  Equation;  Y 

Slope  of  the  cost/quantity  curve  =  0.92 

Slope  of  the  cost/rate  curve  =  0.99 

Multiple  Regression  Coefficient  (R^)  =  0.99 


g.  It  should  be  noted  that  the  multiple  regression 

coefficient  (R^)  is  usually  presented.  R^  is  a  measure  of  the 
"goodness  of  fit"  of  a  given  regression  line.  It  simply 
describes  the  proportion  of  variance  in  the  dependent  vari¬ 
able  that  can  be  attributed  to  the  explanatory  variable(s). 
Since  it  is  a  proportion,  it  is  bounded  to  lie  between  (0) 
and  ( 1 ) .  At  the  extremes ,  a  R^  value  equal  to  zero  would 
imply  that  none  of  the  variation  in  the  dependant  variable 
can  be  described  by  the  regression  line.  When  R‘  equals  (1), 
this  means  that  all  of  the  variation  in  the  dependant 
variable  is  captured  by  the  regression  line.  Thus,  as  the 
value  of  R^  becomes  larger,  more  of  the  variation  in  the 
dependent  variable  is  explained  by  the  regression  line.  The 
general  rule  is  the  larger  the  value  of  R^ ,  the  better  the 
estimating  equation. 

F-6.  CQM  COST  QUANTITY  MODEL 

a.  Ideally,  military  procurement  personnel  would  like  to 
deal  in  a  world  of  unlimited  financial  resources.  There, 
they  could  purchase  every  piece  of  equipment  or  system 
desired.  Analysts  would  only  have  to  compile  a  "shopping 
list"  of  goods  and  set  about  acquiring  them.  But  in  the  real 
world,  equipment  procurement  is  not  so  simple.  The  Army 
functions  in  an  environment  of  limited  resources  (which  will 
probably  shrink  in  future  years).  The  quantity  and  types  of 
equipment  and  systems  obtained  are  largely  restricted  by  the 
funding  decisions  of  the  Executive  Branch  and  Congress. 

b.  Given  this  environment  of  budgetary  constraints, 
analysts  need  tools  to  assist  them  in  deciding  how  much 
equipment  or  how  many  systems  to  purchase.  That  is,  to 
determine  the  optimal  quantity  given  a  certain  budgetary 
constraint.  One  method  of  doing  this  was  developed  by  the  US 
Army’s  Cost  and  Economic  Analysis  Center.  CEAC  developed  a 
model  that  was  constructed  in  the  fcrm  of  a  Lotus  1-2-3 
spreadsheet.  In  application,  this  model  proved  difficult  to 
work  with  due  to  a  "circular  reference"  in  the  spreadsheets 
programming.  This  circular  reference  resulted  in  an  endless 
loop,  which  made  the  model  nearly  impossible  to  copy. 


A  good  discussion  on  R^  can  be  found  in  Kmenta,  Jan,  Elements  of 
Econometrics.  Macmillan  Publishing  Company,  New  York,  1986.  Pages  237-243. 
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c.  Because  of  this  difficulty,  the  CEAC  model  was 
modified  into  the  CQM  presented  below.  The  CQM  is  designed 
to  provide  equipment  production  and  cost  estimates  based  on 
experience  curve  theory.  Estimates  are  provided  in  terms  of 
both  constant  and  current  dollars.  In  cases  where  there  are 
budget  constraints,  the  model  will  compute  how  many  items  may 
be  procured  with  the  constrained  funds.  While  still 
retaining  USACEAC's  original  outline,  CQM  also  provides  the 
following: 

(1)  Experience  curve  parameter  estimation. 

(2)  Current  and  constant  dollar  estimates. 

(3)  A  method  of  providing  for  programmed  and 
unprogrammed  changes  in  costs  through  shifts  to  the 
experience  curve. 

d.  The  CQM  model  is  based  on  the  cost /quantity /rate 
experience  curve  method  of  (7)  above.  That  is, 

7.-^)  Y~AQ-^R-^ 

where  all  variables  are  as  previously  defined.  The  CQM  model 
produc3s  estimates  of  the  experience  curve  parameters.  It 
also  produces  the  experience  curve  percentages  and  other 
descriptive  statistics.  There  is  an  option  to  drop  the 
production  rate  (R)  variable  if  it  is  found  to  be 
statistically  insignificant.  If  that  should  be  the  case,  CQM 
can  estimate  the  experience  curve  without  this  variable. 

e.  Total  production  costs  are  found  by  examining  the  area 
under  the  experience  curve.  The  total  production  cost  is 
determined  by  the  following: 

J^.0.5 

AQ‘’R‘dx 

1-0.5 

where ; 


T'C-  total  production  cost 

F -  unit  number  of  the  first  item  produced 
1-  unit  number  of  the  last  item  produced. 

Equation  (11)  can  be  modified  to  determine  the  total  cost  of 
any  given  lot.  This  is  done  below. 


CEC,  ALM-63-3126-LC.  Page  214, 
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where 


TCg  -  total  production  cost  of  a  given  lot 
F’-  unit  number  of  the  first  item  produced 
L-  unit  number  of  the  last  item  produced. 


f.  Table  F-3  provides  a  listing  of  sample  data  used  to 
construct  and  test  the  results  of  the  CQM.  This  table 
consists  of  8  years  of  production  cost  data.  A  total  of 
3,349  units  are  to  be  purchased  at  a  budgeted  cost  of  $10,810 
million.  Average  unit  cost  is  found  by  dividing  lot  cost  by 
lot  size.  The  cumulative  lot  mid-point  is  computed  by 
equation  (8)  as  presented  above. 

g.  Equation  (7. a)  is  estimated  in  the  usual  manner,  by 
applying 


Table  P-3. 

Cost,  Quantity,  and  Rate  Data  for  a 

Given  Weapons  System 

(cost  data  in  millions  of  dollars) 

Lot  mid- 

Production 

Fiscal  year 

Unit  cost 

(y) 

point  {Q) 

rate  (R) 

1996 

4.995 

29.15 

96 

1997 

3.869 

234.23 

335 

1998 

3.451 

657.84 

500 

1999 

3.225 

1,168.18 

500 

2000 

3.072 

1,672.19 

500 

2001 

2.980 

2.174.34 

500 

2002 

2.923 

2,675.68 

500 

2003 

2.892 

3,137.04 

418 

OLSQ  to  the  log-transformed  variables.  The  average  unit  cost 
variable  ( V )  is  the  dependent  variable,  and  the  explanatory 
variables  are  the  cumulative  lot  mid-point  {Q)  and  production 
rate  (/?).  Table  F-4  displays  the  regression  results  from  the 
data  of  Table  F-3.  Also  included  are  the  results  as  they  are 
presented  by  the  CQM. 
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h.  The  experience  curve  coefficient  estimates  are  then 
utilized  to  determine  the  total  cost  (TC).  This  result  can 
be  seen  in  Table  F-5. 


1  Table 

-  Fr4_- _ 1 

Estimated  Experience  Curve  Parameters 
(based  on  data  from  Table  F-3) 

Regression  output: 

CQM  Model  output : 

First  Unit  Cost  *  7.743 

Constant  =  2.04677426 

QTY  Slope  (b)  -  -0.11404 

b  =  -0.114044 

Cost/Qtv  slope  =  0.9240 

c  »  -0.0118126 

Rate  Slope  (c)  =  -0.01181 

R  squared  =  0.999 

Cost/Rate  slope  =  0.9918 

As  can  be  verified,  the  CQM  estimates  the  total  cost  of  this 
particular  system  to  be  $10,792.5  million.  Presented  also  in 
Table  F-5  are  the  projected  tot«l  costs  of  the  system.  The 
projected  total  cost  amounts  to  $10,810  million.  Comparison 
of  the  model's  estimate  to  the  projected  total  costs  reveals 
that  the  model  forecast  projected  total  costs  quite  well. 

i.  The  portion  of  the  CQM  devoted  to  computing  production 
costs  given  desired  quantities  and  possible  budget  con¬ 
straints  is  divided  into  three  sections.  The  first  is  the 
input  section,  which  requires  the  user  to  enter  certain  key 
data.  This  data  includes; 

(1)  First  Unit  Cost  (A).  It  should  be  noted  that  this 
variable  must  be  based  on  an  annual  production  rate. 12 


(2)  Quantity  slope 

(3)  Rate  slope  (c) 

(4)  Production  rate 


(R)  by  fiscal  year. 


This  portion  of  CQM  does  not  perform  the  regression  estimates  of  equation 
(7  a).  This  must  be  done  in  another  portion  of  the  model,  and  then  manually 
entered. 
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Table  P-5. 

Estimate  of  Total  Cost 
and  Lot  Costs 

(based  on  data  from  Table  F-3) 

Fiscal 

Production 

Pro j  ected 

Model  est. 

year 

rate  (.R) 

lot  cost* 

lot  cost* 

1996 

96 

480 

470.1 

1997 

335 

1,296 

1,295.0 

1998 

500 

1,726 

1,714.5 

1999 

500 

1,613 

1,607.0 

2000 

500 

1,536 

1,542.9 

2001 

500 

1,490 

1,497.4 

2002 

500 

1,462 

1,462.5 

2003 

418 

1,209 

1,203.2 

TOTAL 

3,349 

10,810 

10,792.5 

*In  millions  of  dollars. 


(4)  Previous  quantity  produced. 

(5)  First  fiscal  year  of  procurement 

(6)  Projected  shifts  of  first  unit  cost,  by  fiscal 
year.  These  can  be  positive  or  negative. 

(7)  Desired  quantities,  by  fiscal  year. 

(8)  Budget  constraints,  by  fiscal  year.  These  should 
be  entered  in  terms  of  current  dollars. 

(9)  Inflation  factors,  by  fiscal  year. 

The  second  section  is  the  Desired  Production  at  Maximum 
Rate  Section.  This  section  computes  the  constant  dollars 
needed  to  produce  the  desired  amount  of  items.  Also  calcu¬ 
lated  by  this  section  are  the  number  of  items  that  can  be 
produced  given  the  budget  constraints,  budgeted  funds  against 
estimated  production  costs,  and  whether  there  is  a  surplus  or 
shortage  of  budgeted  funds. 

The  final  section  is  the  Actual  Production  at  Maximum 
Rate  Section.  This  section  will  compute  the  total  number  of 
items  that  can  be  produced  given  the  budget  constraints . 

These  items  will  be  presented  based  on  the  assumption  that 
they  can  be  produced  at  a  rate  equal  to  the  originally 
desired  annual  rate.  In  the  case  of  a  severe  budget 
constraint,  this  would  be  analogous  to  a  manufacturer 
producing  at  top  capacity  for  the  first  few  months  of  the 
fiscal  year  and  then  halting  production  for  the  remainder  of 
the  fiscal  year. 
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F-7.  SPREADSHEET  VERSION  OF  CQM 

a.  The  three  sections  discussed  above  also  appear  in  a 
spreadsheet  version  of  the  CQM.  The  model  depends  upon  the 
user  entering  string  and  formula  data  into  the  spreadsheet. 
This  projects  production  and  budget  information  for  two 
fiscal  years.  Information  on  subsequent  fiscal  years  is 
found  by  copying  the  formulas. 

b.  The  spreadsheet  version  of  CQM  is  operated  in  the 
following  manner: 

(1)  Template  Expansion.  The  model's  template  only 
provides  data  for  2  fiscal  years.  But  it  can  easily  be 
extended  to  account  for  additional  years.  This  is  done  by 
copying  the  spreadsheet's  fourth  column  to  the  «th  column, 
where  n  is  the  number  of  additional  fiscal  years  to  be 
considered. 

(2)  Data  Entry.  The  input  variables  discussed  in 
paragraph  F-6i  above  are  entered. 

c.  The  model  can  be  modified  to  account  for  inflation. 
The  model  assumes  that  future  budget  estimates  are  made  in 
terms  of  then  year  dollars.  If  appropriate  inflation  factors 
are  entered,  then  a  constant  dollar  baseline  can  be  obtained. 
It  should  be  mentioned  that  when  a  baseline  is  chosen,  the 
first  unit  cost  (A)  should  be  expressed  in  terms  of  the 
baseline  dollars. 

d.  The  CQM  also  has  the  ability  to  account  for  signif¬ 
icant  changes  to  recurring  production  costs.  This  is 
accomplished  by  shifting  the  experience  curve. 

e.  The  CQM  provides  the  capability  to  develop  production 
estimates  based  on  budgetary  constraints.  Through  the  use  of 
shifts  in  the  unit  cost  curve,  the  model  provides  the 
capability  to  quickly  assess  the  effects  of  significant 
changes  to  recurring  production  costs. 

F-8.  LIFE  CYCLE  COST  MODEL  (LCCM) 

a.  The  purpose  of  the  LCCM  is  to  examine  the  baseline 
cost  estimate  and  ACP  of  a  weapon  system  by  the  P92  subcate¬ 
gories  within  each  of  the  Big  Five  cost  categories.  The 
values  input  into  the  BCE  are  derived  from  the  cost  histories 
of  weapon  system  programs.  Systems  include  the  following: 
aircraft,  missiles,  air  defense  combat  support,  and  fire 
support,  among  others. 

b.  The  LCCM  is  a  large  spreadsheet-based  model  that  is 
intended  to  recompute  new  values,  e.g.,  due  to  sensitivity 
analysis  on  specific  quantities  and  cost  levels,  current 
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proposed  budget  ceilings  or  revised  contractual  agreements. 
The  recalculated  results  are  based  on  the  ratios  of  the 
historical  production  quantities  that  are  listed  in  the  BCE 
to  the  updated/revised  production  quantity  entries. 

c.  For  example,  given  a  production  rate  of  516  M1A2 
tanks  in  the  fifth  year  at  a  total  cost  of  $1,825  billion,  if 
a  revised  budget  proposal  or  new  production  rate  schedule 
provides  for  a  revised  quantity  of  400  vehicles,  the  CQM 
would  calculate  the  proportional  production  cost  reduction 
(i.e.,400  MlA2s  *  $2.8  million  allocated  to  producing  a 
tank).  The  LCCM  addresses  all  proportional  changes  to  the 
respective  production  subcategories  based  on  the  revised 
quantities  and  production  costs.  All  associated  and 
procurement- funded  cost  subcategories  would  change  in  a 
similiar  manner. 

d.  An  example  of  a  production  cost  used  in  the  LCCM  is 
cost  element  2.02,  recurring  investment  cost.  This  cost  is 
comprised  of  the  following  subcategories:  manufacturing 
cost,  recurring  engineering  cost,  sustaining  tooling  cost, 
and  quality  control  cost.  The  LCCM  is  set  up  so  that  when  the 
data  is  input  at  the  lowest  level  cost  element  (if  avail¬ 
able),  it  automatically  calculates  the  next  highest  cost 
element. 

P-9.  LCCM  OUPUT 

a.  Several  types  of  informarion  that  can  be  derived  from 
the  LCCM.  It  is  assumed  that  entries  are  provided  in  either 
current  or  constant  base  year  dollars  from  either  the  BCE  or 
ACP. 


b.  A  sample  from  the  BCE  data  subcategories  using  the 
M1A2  tank  for  a  current  year  within  the  2.0  category  (without 
subcategories  such  as  2.091  System  Project  Management)  is 
illustrated  in  Table  F-6. 

Table  F-6.  Production  Phase  Breakdown 


Life  Cycle  Cost  Model  —  Production  Phase  Illustration  M1A2 

Tank 

Category 

Cost  Element 

Appropriati 

on 

FY  90  ($000) 

2.0 

Production  Total 

OMA 

PA 

33,189.4 

1,770.5 

31,418.9 

2.01 

Nonrecurring 

PA 

2,820.5 

2.02 

Recurring 

PA 

25,716.0 

nsatmm 

ECOs 

PA 

98.9 

llz.oj _ 

Data 

PA 

177.0 

in 

o 

04 

Sys  Test  &  Eval 

PA 

234.5 
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2.06 

Train  Serv  &  Equipment 

PA 

519.7 

2.07 

Intial  Spares 

OMA 

1,770.5 

00 

o 

• 

CM 

Operate /Site 

PA 

2.09 

Other  Prod  Fund 

PA 

1,852.3 

c.  A  sample  of  the  output  format  showing  the  rolled  up 
costs  for  the  Big  5  cost  categories  over  is  illustrated  below 
in  Table  F-7 .  Total  life  cycle  costs  for  the  weapon  system 
are  also  calculated. 


Table  F-7 .  Sample  Output  for  Big  5  Categories 


POM 

POM 

POM 

POM 

POM 

POM 

1 

2 

3 

4 

5 

6 

Total 

Total 

Total 

Category 

1992 

1993 

1994 

1995 

1996 

1997 

POM 

Outyears 

Life 

Cycle 

Dev 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Prod 

X 

X 

X 

X 

X 

X 

X 

X 

X 

MCA 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Fldq 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Sustain 

X 

X 

X 

X 

X 

X 

X 

X 

X 

d.  Another  form  of  output  is  designed  to  provide  a 
breakdown  of  all  costs  by  appropriation;  OMA,  MPA,  MCA,  and 
PA.  Moreover,  another  depiction  is  provided  by  procurement 
funded  costs  (total  fly  away  cost,  total  weapon  systems  cost, 
total  procurement  cost,  and  total  program  acquisition  cost) 
in  both  constant  and  current  dollars  for  a  given  base  year. 

e.  The  basic  spreadsheet  template  was  developed  using  a 
PC-based  application,  Quattro  Pro.  The  file  is  initially  set 
up  in  protected  mode  so  the  formulas  cannot  be  overwritten; 
however,  an  option  is  available  for  overwriting  the  formulas 
by  changing  the  defau’t  to  unprotected  mode  when  input  data 
is  provided  at  a  higher  level,  e.g.,  2.011  instead  of  2.01. 
The  step  is  for  the  analyst  to  copy  the  template  to  another 
spreadsheet  file  (one  which  has  substantive  meaning  such  as 
MLRS.WQl  or  BFVS.WQl)  so  that  the  basic  formulas  and  layout 
maintain  a  standard  format. 

f.  The  output  of  the  LCCM  provides  several  rollups,  in 
both  current  and  constant  dollars,  with  an  indexed  base  year 
which  is  contingent  on  the  budget  year  for  the  given  system. 
Output  formats  include  total  requirements  by  appropriation 
category.  Big  5  category,  procurement  funded  cost  as  well  as 
a  listing  of  the  systems'  associated  items  of  equipment 
(ASIOE).  Timeframes  of  interest  include  previous  years 
through  the  current  year,  budget  year,  POM  years,  total 
outyears,  and  total  life  cycle  cost. 

g.  Bar  graphs  can  easily  be  called  up  and  viewed  within 
each  weapon  system  spreadsheet.  A  sample  illustration  of  the 
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total  life  cycle  cost  (FY  91  current  dollars)  of  the  single 
channel  ground  and  radio  airborne  system  (SINCGARs)  by  each 
applicable  appropriation  is  illustrated  in  Figure  F-1. 


Figure  F-1.  SINCGARS  Appropriation  Summary 

h.  In  conclusion,  the  LCCM  provides  a  useful  tool  to 
perform  life  cycle  costing  for  a  given  weapon  system.  The 
input  is  based  on  the  BCE;  however,  other  data  can  be  used  if 
desired.  Assuming  all  the  input  values  are  converted  to  fit 
into  the  structure  of  the  spreadsheet  template,  costs  in  many 
modes  of  display  are  calculated.  (Note:  in  order  to 
successfully  use  the  model,  a  286-based  machine  running  at 
12MZ  is  recommended  as  a  minimum  standard  with  available 
expanded  memory  of  at  least  1MB. ) 
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APPENDIX  6 
TOPSIS  USER  NOTES 


G-1.  INTRODUCTION.  The  analytic  tool  selected  for  measure 
of  effectiveness  integration  was  the  "Technique  for  Order 
Preference  by  Similarity  to  an  Ideal  Solution,  better  known  by 
its  acronym,  TOPSIS.  This  technique  was  developed  by  Ching-Lai 
Hwangl  and  Kwangsun  Yoon  at  Kansas  State  University  and  has  been 
widely  used  by  the  Army  and  taught  in  operations  research 
classes  at  the  Army  Logistics  Management  Center. 

G-2.  USER’S  INSTRUCTIONS  FOR  MS-DOS.  To  execute  the 
TOPSIS  software  independent  from  the  capsule  environment,  the 
following  steps  should  be  taken: 

a.  Preparation  of  Input.  An  ASCII  text  file  forms  the 
input  for  the  TOPSIS  calculation  engine.  The  text  file  is  used 
to  mimic  the  relational  tables  found  in  the  data  base 
environment.  Because  of  this  "table"  structure,  the  input  has  a 
particular  structure,  but  is  relatively  free  format  otherwise. 
You  never  have  to  count  columns  or  use  numeric  format  codes. 

The  text  file  may  be  prepared  using  a  normal  text  editor,  but  is 
most  commonly  prepared  with  a  microcomputer  spreadsheet  program 
such  as  Microsoft  Excel.  Since  all  spreadsheets  support  the 
display  of  multiple  rows  and  columns  of  data,  they  are  particu¬ 
larly  advan-ageous  as  a  user  interface.  When  the  spreadsheet 
containing  the  inputs  is  finished,  it  is  only  necessary  to  save 
the  output  as  "text,"  an  option  supported  by  all  known  spread¬ 
sheet  products. 

b.  File  Format.  The  format  for  the  input  file  is  as 
follows : 

(1)  The  first  three  lines  of  the  file  are  "header"  lines 
which  label  the  decision  attributes  associated  with  each 
alternative.  These  three  lines  all  begin  with  a  TAB  character 
(or  a  sequence  of  spaces)  to  distinguish  them  as  header  lines. 

(2)  The  first  line  of  the  file  contains  the  names  of  the 
decision  attributes.  Each  name  can  have  any  number  of 
characters,  but  must  not  contain  any  blanks,  tabs,  carriage 
returns,  or  commas,  as  these  characters  are  used  as  field 
separators . 

(3)  The  second  line  of  the  file  contains  either  a  plus 

sign  ("+")  or  a  minus  sign  indicating  that  the  attribute 


^Hwang,  Ching-Lai  and  Yoon,  Kwangusn,  Multiple  Attri'uute  Decsion  Making- 
Methods _ and  Applications.  Springer-Verlag,  Berlin,  1981. 


G-  1 


CAA-SR-91-9 


in  this  colunm  is  either  a  benefit  or  a  cost  measure 
respectively. 

(4)  The  third  line  of  the  file  contains  the  weight 
associated  with  the  attribute.  This  is  a  number  ^'etween  0  and 
1,  and  is  computed  using  the  methods  described  in  Chapter  7. 

(5)  All  subsequent  lines  after  the  third  line  contain 
rows  of  the  decision  matrix  with  the  scores  associated  with  each 
alternative.  Each  row  begins  with  an  alternative  name  (which 
again  can  contain  no  blanks  or  tab  characters),  followed  by  the 
scores  associated  with  that  alternative  for  each  attribute. 

c.  Command  Line  Invocation.  To  use  the  engine,  simply 
type  the  command  TOPSIS  followed  by  the  data  file  name  of  the 
file  containing  the  input  data.  (On  the  Macintosh,  double  click 
on  the  application  icon.  You  will  be  prompted  for  the  input 
file  name.)  This  input  file  must  end  with  the  extension  ".dat" 
in  order  to  be  found  correctly.  After  TOPSIS  has  computed  its 
results,  a  file  with  the  same  name,  but  with  extension  ".out", 
will  automatically  be  generated.  If  there  are  no  errors,  TOPSIS 
will  operate  silently,  producing  no  further  output  to  the 
command  interface. 

d.  Output  File.  The  output  file  contains  a  mirror  image 
of  the  input  data  with  two  new  columns  in  the  matrix.  The  first 
new  column  is  the  TOPSIS  coefficient,  and  the  second  column  is 
the  relative  rank  among  the  listed  alternatives. 

e.  Example  Input.  The  following  is  a  small  input  file 
(named  "example.dat")  formatted  properly  for  use  with  the 
standalone  version  of  TOPSIS. 

Table  G-1.  Example  Input 


Political 

Risk 

Congressional 

Opinion 

Asset 

Versatility 

and 

Deployability 

Operational 

and 

Technical 

Risk 

+ 

+ 

+ 

WEIGHTS 

0.1282705 

0.2763505 

0.595378 

BASE  CASE 

31 

19 

36 

ALT  1 

29 

36 

27 

ALT  2 

16 

24 

17 

f.  Example  Command  Line:  To  run  TOPSIS  on  the  above 
data,  the  following  command  line  invocation  is  used: 

C:>  tops is  example 

g.  Example  Output.  The  following  is  the  output  file 
generated  by  the  above  input. 
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Table  G-2 .  Example  Output 


EXA«PLE_coef 

EXAMPLE_rank 

BASE  CASE 

31 

19 

36 

0.327866 

2 

ALT  1 

29 

36 

27 

0.895215 

1 

ALT  2 

16 

24 

17 

0.255277 

3 

h.  Possible  Errors.  The  following  are  typical  errors, 
and  the  recommended  corrective  action: 

(1)  "Could  not  open  <filename>  for  input." 

This  error  is  usually  the  result  of  misspelling  the 
input  file  name,  or  not  having  the  appropriate  extension 
(.dat)  on  the  input  file  name.  Also  common  when  the 
input  file  is  in  another  directory  besides  the  current 
directory.  Provide  either  the  complete  path  and 
directory  name  or  "cd"  to  the  appropriate  directory 
before  running  TOPSIS. 

(2)  "Could  not  allocate  r." 

This  error  is  usually  the  result  of  trying  to  run  TOPSIS 
on  a  machine  with  insufficient  memory.  MS-DOS  machines 
have  the  most  restricted  memory;  There  is  little  you 
can  do.  Try  using  a  Macintosh,  which  can  handle  much 
larger  problems.  If  the  problem  persists  on  the 
Macintosh,  turn  multifinder  off. 

(3)  "Invalid  vector  length"  <Next  line  shows  defective 
vector> 


This  error  is  usually  the  result  of  having  too  many  or 
too  few  attribute  scores  in  a  particular  row.  Count 
them  again.  Also,  look  for  blanks  in  the  name  of  the 
alternative. 
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APPENDIX  B 
FOMOA  USER  MOTES 


H-1.  INTRODUCTION 

a.  The  analytic  tool  selected  for  development  of  the 
acquisition  strategy  was  the  Force  Modernization  Analyzer, 
better  known  by  its  acronym,  FOMOA. .  It  has  been  widely  used  by 
the  Army  for  the  production  of  weapon  system  modernization 
plans.  It  is  based  upon  the  concept  of  linear  programming  and 
econometric  modeling  of  cost-quantity  relationships.  The  Value 
Added  FOMOA  formulation  considers  production,  TOA,  and  force 
structure  constraints  in  conjunction  with  cost  and  benefit  data 
in  order  to  determine  an  optimal  mix  of  weapon  system 
procurements  by  year. 

H-2.  GENERAL  FORM  OF  THE  OPTIMIZATION  MODEL 

a.  The  Formulation.  Documentation  for  the  general 
formulation  of  the  Value  Added  FOMOA  is  provided  within  this 
paragraph  of  the  appendix. 

b.  Key  Words.  The  model  formulation  contains  the 
following  key  words;  SETS,  ENDSETS,  MODEL,  DATA,  ENDDATA,  and 
END.  The  formulation  begins  following  the  SETS  section.  The 
data  files  are  listed  near  the  end  of  the  program  after  the  key 
wor  DATA.  Figure  H-1  shows  the  relative  arrangement  of  the 
major  sections  of  the  formulation  file.  The  formulation  file 
name  ends  with  the  extension  "<f ilename>.mod" . 


SETS 

MODEL 

DATA 


Figure  H-1.  FOMOA  Formulation  Sections 

The  general  form  for  each  of  the  sets  listed  in  the  subsequent 
paragraphs  of  this  appendix  is;  set  name,  elements  of  the  set, 
and  finally  attribute(s)  for  the  elements.  For  example: 

T/1..12/;TOA; 

T  =  Set  Name 
/1..12/  =  Elements 
:TOA  =  Attribute. 
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c.  Comments,  Etc.  Anything  between  an  exclamation  mark 
( ! )  and  a  semicolon  ( ; )  is  a  comment.  FOMOA  does  not 
distinguish  between  lower  and  upper  case.  Index  variables 
(I,J,K,L  below)  could  be  typed  in  lower  case,  but  in  the  output, 
they  would  be  in  upper  case.  Value  Added  FOMOA,  except  for 
comments,  is  written  entirely  in  upper  case. 

d.  Time.  The  first  lines  of  the  SETS  section  contain  the 
following  statement; 


T/1..12/;TOA: 

(1)  The  elements  1,  2,  ...12  of  the  set  T  are  indicated 
between  the  two  slash  (/)  delimiters.  In  terms  of  mnemonics,  T 
is,  of  course,  time.  (In  this  formulation.  Value  Added  FOMOA  is 
currently  executed  using  only  12  planning  years.  In  the  event 
of  a  larger  implementation,  only  two  lines  of  code  need  be 
changed — this  set  line  and  the  line  in  the  DATA  section 
accessing  a  12-year  data  file.) 

(2)  After  the  colon  (:),  attributes  of  the  elements  are 
listed.  (The  semicolon  (;)  indicates  the  end  of  the  statement.) 
Attributes  pertain  either  to  input  data  or  program  variables. 

The  values  for  data  attributes  appear  in  an  input  data  file 
associated  with  each  attribute  and  named  "<attribute  name>.dat". 
The  TOA  attribute  data  would  be  in  a  file  called  "TOA.dar"  and 
the  attribute  has  the  following  definition: 

TOA(I)  Funding  level  for  procurement  for  year  I. 

e.  Time  and  Mission.  The  next  two  statements  define  the 
set  M  of  vehicle  missions/roles/force  packages  which  vary  with 
time.  It  should  be  noted  that  while  the  formulation  explicitly 
accounts  for  missions/roles/force  packages,  in  Phase  I  of  Value 
Added  Analysis,  we  considered  only 
one  mission  per  system. 

M  /TK.  IN.  AT.  EN.  AR.  AM/ :  ; 

TM(T.M)  :  EQM,  WM,  RM,  LM,  UM; 

(1)  The  variable  attributes  of  TM  are; 

EQM(I,J)  The  size  in  terms  of  number  of  vehicles  in 
year  I  of  mission  J  (of  all  types  K — see  below) . 

WM(I,J)  The  total  modernization  weight  of  the  vehicles 
in  year  I  in  mission  J  (see  definitionof  the  modernization 
weight  W  of  vehicle  types  of  the  Time,  Mission,  and  Equipment 
paragraph  below) . 
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(2)  The  data  attributes  of  TM  are; 

RM(I,J)  Minimum  fraction  modernization  required  in 
year  I  in  mission  J  (weight  divided  by  size — see  mission 
constraints  section  below) . 

LM(I,J)  Lower  limit  on  number  of  vehicles  in  year  I  in 
mission  J  (i.e.,  total  number  of  required  vehicles  in  year  I  in 
mission  J) . 

UM(I,J)  Upper  limit  on  number  of  vehicles  in  year  I  in 
mission  J. 

f.  Mission  and  Equipment.  The  next  several  lines  asso¬ 
ciate  each  equipment  type  with  a  mission  and  give  attributes  of 
equipment  that  do  not  vary  with  time.  (They  involve  planning 
years  1  or  2 . ) 

E  /  (g)file(equip.dat)  / : ; 

ME  (M,E)  /  (2)file(mission.dat)/ ;  ST,  FE1,  FE2; 

(1)  ME  is  a  subset  of  the  direct  product  of  M  and  E. 

The  definition  of  ME  is  given  by  the  parenthetical  expression 
(M,E)  and  a  list  of  specified  elements  of  between  the  slash 
(/)  delimiters. 

(2)  The  attributes  of  ME  are  all  of  the  data  category. 
They  are: 

ST(J,K)  Inherited  assets  available  for  year  1 
(starting  year)  in  mission  J  of  type  K. 

FE1(J,K)  Programmed  fielding  in  year  1  (programmed 
buys  2  years  before  planning  period)  for  vehicles  in  mission  J 
of  type  K. 

FE2(J,K)  Programmed  fielding  in  year  2  (programmed 
buys  1  year  before  planning  period)  for  vehicles  in  mission  J  of 
type  K. 


g.  Time,  Mission,  and  Equipment.  The  next  line  of  the 
code  defines  the  set  TME  and  is  the  capstone  of  what  has  gone 
before.  It  gives  the  equipment  attributes  that  can  vary  with 
time. 


TME  (T.ME)  :  X,Y,EQ,  CP,CO,W,LX,UX,UE; 

(1)  The  variable  attributes  of  TME  are: 

X(I,J,K)  Number  of  vehicles  purchased  in  year  I  in 
mission  J  of  type  K. 
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Y(I,J,K)  Number  of  vehicles  retired  in  year  I  in 
mission  J  of  type  K. 

EQ(I,J,K)  The  number  of  vehicles  in  year  I  in  mission 
J  of  type  K  (force  composition). 

(2)  The  data  attributes  are: 

CP(I,J,K)  Purchase  cost  of  a  vehicle  in  year  I  in 
mission  J  of  type  K. 

CO(I,J,K)  O&S  cost  of  a  vehicle  in  year  I  in  mission  J 
of  type  K. 


W(I,J,K)  Modernisation  weight  of  a  vehicle  in  year  I 
in  mission  J  of  type  K.  This  weight  is  a  user  input  value 
between  0  and  1.  Fully  modernized  vehicles  have  weight  1, 
completely  obsolete  vehicles  have  weight  0. 

LX(I,J,K)  Lower  limit  on  number  of  vehicles  procured 
in  year  I  in  mission  J  of  type  K. 

UX(I,J,K)  Upper  limit  on  number  of  vehicles  procured 
in  year  I  in  mission  J  of  type  K. 

UE(I,J,K)  Upper  limit  on  number  of  vehicles  in  year  I 
in  mission  J  of  type  K.  (Used  only  in  minimum  cost  runs  to 
remove  old  systems  with  low  operation  and  support  costs.) 

h.  Production  Lines.  The  next  several  lines  name  the 
production  lines,  specify  the  equipment  type  each  produces,  and 
give  the  line  attributes,  all  of  which  can  vary  with  time.  It 
should  be  noted  that  while  the  formulation  explicitly  accounts 
for  multiple  production  lines  per  system,  in  Phase  I  of  Value 
Added  Analysis,  only  one  production  line  per  system  was 
considered. 

P.  /(a)file(lines.dat)/ ;  ; 

PME  (P,M,E)  /  (a)file(PME.clat)/; 

TP  (T.P)  :  XP,LP,UP,Z; 

The  variable  attribute  of  TP  is: 

XP(I,L)  Total  number  of  vehicles  produced  in  time  I  on 

line  L. 

The  data  attributes  of  TP  are: 

LP(I,L)  Lower  annual  production  limit  in  year  I  for 

line  L. 
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UP(I,L)  Upper  annual  production  limit  in  year  I  for 

line  L. 

Z(I/L)  Open/close  production  line  status  in  year  I  for 
production  line  L.  If  the  line  is  open,  the  status  value  is  1; 
0,  otherwise. 

i.  Dependent  and  Decision  Variable 

(1)  The  variables  EQM  and  WM  of  set  TM  are  both  defined 
in  term  of  the  variable  EQ  of  set  TME  as  follows: 

[H.01] 


EQM(U)  =  2  EQ(1,J.K)  for  l€T,  JEM. 

X  i*  JIR  lEHS 

LH.02] 


WM(I,J)  =  ^  EQ(I,J.K)*W(I,J,K)  for  I  e  T,  J  E  M. 

X  KAX  (EKB 


(2)  The  variable  EQ  is  in  turn  defined  in  term  of  the 
more  basic  variables  X  and  Y  of  TME  as  follows: 

For  (J,K)  E  ME  and  I  >  2 

EQ{1,J,K)  =  ST{J,K)  +  FE1(J,K)  -  Y{1.J.K) 

EQ{2,J,K)  =  EQ(1.J,K)  +  FE2(J.K)  -  Y(2,J,K) 

EQ(I,J,K)  =  EQ(I-1,J,K)  +  X(I-2.J,K)  -  Y(U,K) 

Note  that  vehicles  pro  :ured  in  year  1-2  are  fielded  in  year  I 
because  of  a  2 -year  lag  between  the  purchase  year  and  the 
operational  year. 

(3)  The  variable  XP  is  defined  in  terms  of  the  variable 
X  as  follows: 

[H.03] 


XP{1,L)  =  2  Z(».l-rX(S,J,K)  for  1  ET 

(  L.  JX  IS  PKE 

(4)  The  variables  EQM,  WM,  EQ,  and  XP  can  be  considered 
as  functions  of  the  decision  variables  X  and  Y.  There  is  one 
other  decision  variable,  TMSTAR.  It  can  be  thought  of  as  a 
technical  maximizer.  It  occurs  in  the  maximization  objective 
function  and  in  a  set  of  six  constraints.  These  constraints 
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maximize  the  minimum  of  the  six  ratios  EQM(5,J)  to  LM(5,J) 
involving  the  target  (5th)  planning  year  and  the  six  missions  J. 

j  .  Objective  Function 

(1)  Objective  Function  for  the  Minimum  Cost 
Version.  This  objective  function  minimizes  the  sum  of  all 
procurement  costs  and  all  operations  and  support  costs  over  the 
5  year  planning  period.  It  is  given  by: 

[H.04] 


Minimize  2|cp*X  +  C0*EQ 

THE 

(2)  Objective  Function  for  the  Maximum 
Modernization  Version.  This  objective  function  maximizes 
the  sum  of  the  technical  maximizer  times  1000  and  the  annual 
total  mission  modernization  weights  WM.  Any  relatively  large 
number  can  be  used  in  place  of  1000.  It  is  given  by: 

[H.05] 


Maximize  1000  *TMSTAR 


k.  Constraints 


(1)  Funding  Constraints.  The  following  provides 
annual  funding  limits  on  procurement  dollars: 

[H.06] 


2  CP(I,J,K)*X(I.J,K) )  <  TOA(I)  for  I  6  T. 

(.iK  )  €HE 


(2)  Production  Line  Constraints.  The  bounds 
statement  on  XP  in  the  code  provides  the  following  annual  lower 
and  upper  limits  on  the  production  lines: 

LP(l,J)s  XP(I.J)  5  UP{I,J) 
for  I  €  T.  L  e  P. 

(3)  Mission  Constraints 

(a)  Minimum  total  requirements  by  mission.  The 
bounds  statement  on  EQM  in  the  code  provides  the  following: 

LM(I,J)  s  EQM(I.J)  s  UM(I,J)  for  I  6  T,  J  6  M. 
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(b)  Minimum  modernization  fraction  requirements  by 
mission.  The  following  enforce  a  minimum  mission  weight  to 
mission  size  ratio: 

WM(I,J)  a  RM(I,J)*EQM(I.J)  for  I  ST,  J  £  M. 


(4)  Constraints  on  Vehicle  Types  (Other  than  Sums 
over  Vehicle  Types) 

(a)  Constraints  on  retirement  decision  variables  for 
all  vehicle  types.  The  following  ensure  that  no  more  vehicles 
of  a  given  type  are  retired  than  are  in  the  force  the  previous 
year. 


Y(1,J.K)  s  ST(1.J,K)  for  (J.K)  £  ME 
Y(I.J.K)  iEQ(l-1,J.K)  for  (J,K)  £MEand  I  >  1. 


(b)  Constraints  on  number  of  vehicles.  The  following 
can  be  used  to  retire  old  vehicle  types: 

EQ(I,J,K)  5  UEO.J.K)  for  (l,J,K)  £TME. 


(5)  Technical  Constraints.  These  constraints  are 
only  for  maximization  runs.  See  discussion  of  the  decision 
variable  TMSTAR  and  of  the  maximization  objective  function 
'uove.  (Multiplication  by  100  is  done  below  for  scaling 
purposes  to  counteract  the  large  denominator  involving  LM. ) 

TMSTAR  s  100*WM(S12E(T),J)  /  LM(SlZE(T),J)  for  J  £  M. 


(6)  Procurement  Constraints.  The  bounds  statement 
on  X  provides  the  following: 

LX(I,J,K)  s  XO.J.K)  s  UX(I,J,K) 
for  (I, J.K)  £TME. 


H-3.  USER  S  INSTRUCTIONS  FOR  MS-DOS.  To  execute  the 
FOMOA  software  independent  from  the  capsule  environment,  the 
following  steps  should  be  taken: 

a.  Preparation  of  Input.  A  series  of  ASCII  text  files 
form  the  input  for  the  FOMOA  calculation  engine.  Each  text  file 
is  used  to  mimic  a  relational  table  found  in  the  database 
environment.  Because  of  this  "table"  structure,  the  input  has  a 
particular  structure,  but  is  relatively  free  format  otherwise. 
You  never  have  to  count  columns  or  use  numeric  format  codes. 

The  text  file  may  be  prepared  using  a  normal  text  editor,  but  is 
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most  commonly  prepared  with  a  microcomputer  spreadsheet  program 
such  as  Microsoft  Excel.  Since  all  spreadsheets  support  the 
display  of  multiple  rows  and  columns  of  data,  it  is  particularly 
advantageous  as  a  user  interface.  When  the  spreadsheet 
containing  the  inputs  is  finished,  it  is  only  necessary  to  save 
the  output  as  "text,"  an  option  supported  by  all  known 
spreadsheet  products. 

b.  File  Format.  The  format  for  all  input  files  is 
generally  as  follows: 

(1)  Each  input  file  is  either  a  matrix  or  a  vector  (see 
Table  H-1).  At  the  beginning  of  the  file  are  a  series  of 
optional  "header"  lines  which  label  the  vector  or  matrix  column 
associated  with  each  alternative.  The  header  lines  all  begin 
with  a  TAB  character  (or  a  sequence  of  spaces)  to  distinguish 
them  as  header  lines.  There  may  be  zero  or  any  number  of  header 
lines  in  each  file. 


Table  H-1.  Input  Files  by  Type 


Input  File  Name 

EBSESEHM 

Data  Content 

E  -  0FILE(eq  liat.dat) 

Vector 

List  of  Equipment  Names 

TOA  -  @ FILE (toa.dat); 

Vector 

Accumulative  Dollars  By  Year 

CP  -  9FILE(cp.dat) ; 

Vector 

Procurement  Dollars  By  System 

CO  «  9FILE(co.dat) ; 

Vector 

O&S  Dollars  By  System 

LX  -  »FILE(lx.dat); 

Vector 

Lower  Production  Limit  By  System 

UX  -  «FILE(ux.dat); 

Vector 

Upper  Production  Limit  By  System 

LM  -  «FILE(lm.dat); 

Vector 

Lower  Force  Structure  Limit  By  System 

OM  -  «FILE(um,dat) ; 

Vector 

Upper  Force  Structure  Limit  By  System 

ST  -  ?FILE(st.dat) ; 

Vector 

Starting  Inventory  By  System 

FEl  -  eFILE(fel.dat) ; 

Vector 

1st  Year  Previous  Fieldings 

FE2  »  @FILE(fe2.dat) ; 

Vector 

2nd  Year  Previous  Fieldings 

Z  -  eFILE(z.dat) ; 

Matrix 

Production  Line  Start/Stop  By  Year 
By  System 

VA  -  @FILE(va.dat) ; 

Matrix 

Value  Added  Coefficent  By  Year  By 
System 

(2)  All  subsequent  lines  after  the  header  lines  contain 
elements  of  the  vector  or  rows  of  the  matrix  associated  with 
each  alternative.  Each  row  begins  with  an  alternative  name 
(which  can  contain  no  blanks  or  tab  characters),  followed  by  the 
values  associated  with  that  alternative. 

(3)  Each  data  file  has  a  particular  neune  associated  with 
the  name  of  the  vector  or  matrix  in  the  formulation.  The  name 
itself  is  not  hard-coded,  but  is  specified  by  name  in  the 
formulation  source  file.  For  exaunple,  in  the  formulation  there 
is  a  statement  to  associate  the  vector  variable  "TOA"  with  the 
data  file  "toa.dat"  as  follows: 

TOA  =  @FILE( toa.dat ) ; 
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c.  Command  Invocation.  To  use  the  FOMOA  engine,  simply 
double  click  on  the  application  icon.  You  will  be  prompted  for 
the  input  file  name.  This  input  file  must  end  with  the 
extension  ".mod"  in  order  to  be  found  correctly.  After  FOMOA 
has  computed  its  results,  a  file  with  the  same  name,  but  with 
extension  ".out"  will  automatically  be  generated  in  the  same 
folder.  If  there  are  no  errors,  FOMOA  will  operate  silently 
producing  no  further  output  to  the  command  interface. 

d.  Output  File.  The  output  file  contains  a  detailed 
listing  of  the  values  associated  with  each  decision  variable  in 
the  linear  programming  formulation.  A  table  of  output  values 
lists  the  number  of  systems  to  be  procured  by  year  for  each  year 
of  the  planning  horizon.  The  number  of  years  in  the  planning 
horizon  is  specified  in  the  problem  formulation  source  file. 
Table  H-2  provides  the  output  for  the  A3  case  study. 

Table  H-2.  Example  Output 


System 

1992 

1993 

1994 

1995 

1996 

1997 

1998 

1999 

2000 

2001 

2002 

Lower 

Upper 

SADARM 

513 

391 

382 

360 

360 

360 

360 

360 

360 

360 

513 

A.MSH 

0 

240 

240 

180 

180 

180 

180 

180 

180 

180 

180 

180 

240 

NLOS 

0 

0 

16 

16 

16 

16 

16 

16 

16 

0 

0 

0 

16 

M1A2 

0 

0 

0 

64 

64 

64 

64 

64 

64 

0 

0 

0 

318 

BRADLEY 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

198 

e.  Possible  Errors.  The  following  are  typical  errors, 
and  the  recommended  corrective  action: 

(1)  "Could  not  open  <filename>  for  input." 

Usually  the  result  of  misspelling  the  input  file  name, 
or  not  having  the  appropriate  extension  (.dat)  on  the 
input  file  name.  Also  common  when  the  input  file  is  in 
another  directory  folder  besides  the  current  directory. 
Provide  either  the  complete  path  and  folder  name  or 
place  the  FOMOA  icon  in  the  appropriate  directory 
before  running  FOMOA. 

(2)  "Could  not  allocate  xxx." 

Usually  the  result  of  trying  to  run  FOMOA  on  a  machine 
with  insufficient  memory.  MS-DOS  machines  have  the  most 
restricted  memory  and  there  is  little  you  can  do.  Try 
using  a  Macintosh,  which  can  handle  much  larger 
problems.  If  the  problem  persists  on  the  Macintosh, 
turn  multifinder  off. 
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(3)  "Invalid  vector  length."  <Next  line  shows 
defective  vector> 

Usually  the  result  of  having  too  many  or  too  few 
attribute  scores  in  a  particular  row.  Count  them 
again.  Also,  look  for  blanks  in  the  name  of  the 
alternative. 

(4)  "Invalid  formulation  statement."  <Next  line 
shows  defective  statement> 

Usually  the  result  of  a  syntax  error  in  the  original 
problem  formulation.  Error  recovery  for  formulation 
syntax  errors  is  not  sophisticated.  One  error  may 
generate  (and  most  often  does)  numerous  error 
statements.  Some  common  syntax  errors  include: 

Failure  to  end  each  statement  with  a  semicolon. 
Failure  to  declare  an  index  or  variable  naime  prior 
to  use. 

Unmatched  parenthesis. 

Unrecognized  arithmetic  operator. 

Incorrect  number  of  arguments  in  a  formulation 
language  function. 

—  Failure  to  include  or  use  an  inappropriate  index 
for  a  decision  set. 
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APPENDIX  I 
SPONSOR'S  COMMENTS 


DACS-DPA  4  December  1991 

MEMORANDUM  FOR  US  ARMY  CONCEPTS  ANALYSIS  AGENCY 

SUBJECT:  Value  Added  Analysis  (VAA)  9C-97  Study  Critique 

1.  The  following  report  gives  comments  regarding  your  draft 
report  #CAA-SR-91-9 ,  dated  October  1991,  titled  Army  Program 
Value  Added  Analysis  90-97  (VAA  90-97)  Study: 

a.  There  were  no  specific  editorial  comments. 

b.  All  key  issues  planned  for  analysis  were  adequately 
addressed  in  the  report. 

c.  The  methodology  used  to  conduct  the  study  was 
adequate . 


d.  This  study  will  be  useful  to  this  organization  by 
being  a  primary  source  document  in  formulating  questions  for 
analysis  and  in  answering  specific  questions  regarding  the 
methodology. 

e.  I  believe  that  this  study  is  a  good  initial  report 
on  the  VAA  methodology. 

2.  I  hope  that  these  comments  are  helpful.  Please  feel 
free  to  direct  any  questions  regarding  this  critique  to 
myself  or  to  MAJ  Carlton  in  my  office,  phone:  (703)  225- 
7737,  or  (AV)  695-7737. 


■’OBERT  B.  CLARKE  ^ 

Coj-onel,  GS 

Chief,  Acquisition  and  Support 
Program  Analysis  Division 
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APPENDIX  J 
DISTRIBUTION 


Addressee 


No.  of  copies 


Commander  1 

Army  Research  Institute 

ATTN:  Security  Manager 

5001  Eisenhower  Avenue 

Alexandria,  VA  22333 


Defense  Technical  Information  Center  2 

ATTN:  DTIC-FPS 

Cameron  Station 
Alexandria,  VA  22314-6145 


U.S.  Army  Service  Center  for  the  Armed  Forces  1 

The  Pentagon  Library  (Army  Studies  Section) 

ATTN:  ANRAL-RS/Security  Officer 

Room  1A518,  The  Pentagon 
Washington,  DC  20310-6000 


Director  10 

Program  Analysis  and  Evaluation 

Office  of  the  Chief  of  Staff  of  the  Array 

ATTN:  DACS-DPZ 

Washington,  DC  20310 


Deputy  Chief  of  Staff  for  Operations  and  Plans  10 

Headquarters ,  Department  of  the  Army 
ATTN:  DAMO-ZD 

Washington,  DC  20310 


Deputy  Chief  of  Staff  for  Operations  and  Plans  2 

Headquarters,  Department  of  the  Army 
ATTN:  DAMO-FD 

Washington,  DC  20310 
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Deputy  Chief  of  Staff  for  Operations  and  Plans  2 

Headquarters,  Department  of  the  Army 
ATTN:  DAMO-FDR 

Washington,  DC  20310 

Commander  1 

US  Army  Training  and  Doctrine  Command 

ATTN:  ATCD-P 

Fort  Monroe,  VA  23651-5000 

Commander  1 

TRADOC  Analysis  Command 
ATTN:  ATRC-F 

Fort  Leavenworth,  KS  66027-5200 

Director  1 

RPD,  HQ  TRAC 

ATTN:  ATRC-RP 

Fort  Monroe,  VA  23651-5143 

US  Military  Academy  2 

Department  of  Systems  Engineering 
West  Point,  NY  10996 


Internal  Distribution: 

Reference  Copy: 

Unclassified  Library  2 

Record  Copy; 

Originating  office  (CSCA-FSR)  35 
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GLOSSARY 

ABBREVIATIONS, 

ACRONYMS,  AND  SHORT  TERMS 

A3 

armor  antiarmor 

AAO 

Army  authorization  objective 

AGP 

Army  cost  position 

AHP 

analytical  hierarchy  process 

ALM 

algebraic  lot  mid-point 

AMC 

US  Army  Materiel  Command 

AMM 

Army  modernization  memorandum 

ARC  PS 

Army  Capabilities  Study 

ARIM 

Arm.y  Resource  Integration  and  Management  (study) 

ARSTAF 

Army  Staff 

ASARDA 

Assistant  Secretary  of  the  Army  for  Research, 
Development,  and  Acquisition 

ASIOE 

associated  item(s)  of  equipment 

ASM 

armor  systems  modernization 

BCE 

baseline  cost  estimate 

BFS 

Blue  force  surviving 

BTU 

British  thermal  unit 

CAA 

US  Army  Concepts  Analysis  Agency 

CAB 

cost  analysis  brief 

CEAC 

US  Army  Cost  and  Economic  Analysis  Center 

CECOM 

Communications  and  Electronics  Command 

CER 

cost  estimating  relationship 

CERL 

US  Army  Construction  Engineering  Research 
Laboratory 

CFV 

cavalry  fighting  vehicle 

Cl 

consistency  index 
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CINC 

COFM 

CORBAN 

CORMO 

CQM 

CR 

CY 

DA 

DG 

DM 

DOD 

DPAE 

EBR 

EEA 

EUL 

FAS 

FER 

FOMOA 

FORSCOM 

FR 

FY 

GPS 

hex 

km 

HQDA 

IPR 

IPS 

Glossary-2 


commander  in  chief 

correlation  of  forces  and  means 

Corps  Battle  Analyzer 

Corps  Model 

Cost  Quality  Model 

consistency  ratio 

calendar  year 

Department  of  the  Army 

Defense  Guidance 

decisionmaker ( s ) 

Department  of  Defense 

Director,  Program  Analysis  and  Evaluation 

effective  battalions  remaining 

essential  element (s)  of  analysis 

economic  useful  life 

Force  Accounting  System 

force  exchange  ratio 

Force  Modernization  Analyzer 

Forces  Command 

force  ratio 

fiscal  year 

global  positioning  systems 

hexagon 

kilometer(s) 

Headquarters,  Department  of  the  Army 
in-process  review 
Illustrative  Planning  Scenario 
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JCS 

Joint  Chiefs  of  Staff 

LCC 

life  cycle  cost 

LCCM 

Life  Cycle  Cost  Model 

LER 

loss  exchange  ratio 

LIN 

line  item  number 

LP 

linear  prograimming 

LRRAMP 

Long-Range  Research  and  Development 
Modernization  Plan\ 

m 

meter ( s ) 

MA 

mission  accomplishment 

MACOM 

major  Army  command 

max 

maximum 

mBtu 

millions  of  British  thermal  units 

MCA 

military  construction.  Army 

MDEP 

Management  Decision  Package 

MFCM 

movement  force  center  of  mass 

MILP 

mixed  integer  linear  programming  model 

min 

minimum 

MISM 

major  item  system  map 

MOE 

measure (s)  of  effectiveness 

MSR 

minimum  sustaining  rate 

NDR 

nominal  discount  rate 

NATO 

North  Atlantic  Treaty  Organization 

NCE 

noncombat  effectiveness 

NTC 

National  Training  Center 

OCSA 

Office  of  the  Chief  of  Staff,  Army 
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ODCSOPS 

OLSQ 

OS.M 

OMA 

0MB 

OPLAN 

ORSA 

O&S 

OSD 

PAED 

PEG 

PEO 

PK 

PM 

PO 

POM 

PPBES 

PV 

RAM 

RDA 

RDT&E 

SER 

SIAM 

SLEP 

SME 

SOF 


Office  of  the  Deputy  Chief  of  Staff  for 
Operations  and  Plans 

ordinary  least  squares  regression 

operation  and  maintenance 

operation  and  maintenance.  Army 

Office  of  Management  and  Budget 

operation  plan 

operations  research  and  analysis 

operation  and  sustainment 

Office  of  the  Secretary  of  Defense 

Program  Analysis  Evaluation  Directorate 

Program  Evauation  Group 

Program  Executive  Office 

probability  of  kill 

program  manager 

procurement  objective 

Program  Objective  Memorandum 

Program  Planning  Budgeting  and  Execution  System 
photovoltaic 

reliability,  availability,  and  maintainability 

research,  development,  and  acquisition 

research,  development,  testing,  and  evaluation 

system  exchange  ratio 

secondary  impact  analysis  modifiers 

service  life  extension  program 

subject  matter  expert 

Special  Operations  Forces 
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SSPK 

single  shot  probability  of  kill 

TACOM 

Tank-Automotive  Command 

TAEDP 

Total  Army  Equipment  Distribution  Program 

TAP 

The  Army  Plan 

TC 

total  cost 

TGW 

terminally  guided  weapon 

TOA 

total  obligation  authority 

TOE 

tablet s)  of  organization  and  equipment 

TOPS IS 

Technique  for  Order  Preference  by  Similarity  to 
Ideal  Situation 

TRAC 

TRADOC  Analysis  Center 

TRAC-FLVN 

TRADOC  Analysis  Center,  Fort  Leavenworth 

TRADOC 

US  Army  Training  and  Doctrine  Command 

USAREUR 

US  Army,  Europe 

VAA 

Value  Added  Analysis 

VIC 

Vector-In-Commander 

WESTCOM 

Western  Command 

Wh 

watt-hours 
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ARMY  PROGRAM  VALUE  ADDED 
ANALYSIS  90-97 
(VAA  90-97) 

STUDY 

SUMMARY 

CAA-SR-91-9 

1  ,  ,  .  . 

THE  REASON  FOR  PERFORMING  THE  STUDY  was  to  provide  the  Director  for 
Program  Analysis  and  Evaluation,  and  the  Deputy  Chief  of  Staff  for  Operations 
and  Plans  (DCSOPS)  an  analytical  methodology  and  capability  to  support  the 
development  of  a  balanced  and  effective  Array  Program. 

THE  STUDY  SPONSORS  are  the  Director  for  Program  Analysis  and  Evaluation 
(DPAE),  Office  of  the  Chief  of  Staff,  Army,  and  the  Technical  Advisor,  Office 
of  the  Deputy  Chief  of  Staff  for  Operations  and  Plans  (ODCSOPS),  Head¬ 
quarters,  Department  of  the  Army  (HQDA). 

THE  STUDY  OBJECTIVES  were  to; 

(1)  Formulate  an  analytic  method  for  estimating  the  marginal  value  of 
competing  management  decision  packages  (MDEP)  to  the  Army.  The  methodology 
will  provide  a  common  understandable  basis  for  the  analysis  of  affordability 
issues. 

(2)  Identify  or  develop  prototype  models  that  support  the  Value  Added 
Analysis  methodology  and  provide  management  tools  for  the  analytic  method 
developed. 

(3)  Establish  within  DPAE  and  ODCSOPS  an  in-house  capability  to  conduct 
program  issue  tradeoffs. 

(4)  Conduct  a  proof  of  concept  analysis  during  the  building  of  the  92-97 
Program  Objective  Memorandum  (POM). 

THE  SCOPE  OF  THE  STUDY  included  the  research,  development,  and  acquisition 
(RDA)  appropriations  for  selected  items  of  equipment  in  fiscal  year  (FY)  1999 
and  FY  2004. 

THE  MAIN  ASSUMPTION  of  this  study  is  that  HQDA  needs  a  relatively  quick 
method  for  conducting  program  tradeoffs  which  has  sound  analytical 
underpinnings. 

THE  BASIC  APPROACH  of  this  study  was  to: 

(1)  Identify  and  develop  an  analytical  approach  to  evaluate  program  issue 
tradeoffs. 

(2)  Develop  a  capability  for  implementing  the  methodology  to  include 
software  modules  where  appropriate. 


(3)  Demonstrate  the  methodology  and  capability  by  using  issues  from  the 
90-97  POM  issue  cycle. 

THE  PRINCIPAL  FINDINGS  of  the  study  were: 

(1)  Two  generalized  categories  of  measures  of  effectiveness  (explicit  and 
implicit)  were  determined  to  be  important  in  judging  relative  value. 

(2)  A  linear  combination  of  value  components  appears  to  be  useful  in 
creating  a  single  measure  of  an  investment's  marginal  value. 

(3)  VAA  uses  a  system  of  judgment  weights  which  measures  an  investment's 
value  in  a  dual  context. 

(4)  The  VAA  methodology  as  demonstrated  holds  promise  as  the  decision 
analysis  framework  that  should  be  used  for  conducting  Army  program  tradeoff 
analyses.  Further  work  in  the  areas  of  quick  turnaround  combat  modeling, 
dynamic  costing,  and  data  collection  need  to  be  conducted. 

(5)  A  highly  aggregated  combat  model  can  provide  useful  insights  for 
answering  macro-level  program  tradeoff  questions  and,  to  a  more  limited 
degree,  is  capable  of  providing  specific  program  guidance. 

(6)  In  order  for  aggregated  combat  models  to  be  useful,  they  must  be 
calibrated  to  more  detailed  models  and  data. 

(7)  HQOA  has  a  perspective  which  does  not  include  the  Training  and 
Doctrine  Command's  (TRADOC)  blueprint  of  the  battlefield  per  se.  Their 
headquarters  is  organized  along  the  lines  of  Planning,  Programming, 

Budgeting,  and  Estimation  System  (PPBES)  functions,  and,  although  the 
headquarters  has  some  overlap,  the  specific  thrust  is  the  generation  of 
forces  through  planning  and  programming. 

(8)  The  cost-benefit  analysis  in  the  form  of  the  optimization  model 
requires  further  research  into  the  use  of  dynamic  costing  of  the  procurement 
costs. 

THE  STUDY  EFFORT  was  directed  by  LTC  James  A.  Richmann,  Force  Systems 
Directorate,  US  Army  Concepts  Analysis  Agency  (CAA). 

COMMENTS  AND  QUESTIONS  may  be  sent  to  the  Director,  US  Army  Concepts 
Analysis  Agency,  ATTN:  CSCA-FSR,  8120  Woodmont  Avenue,  Bethesda,  Maryland 
20814-2797. 
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Doctrine  Command's  (TRADOC)  blueprint  of  the  battlefield  per  se.  Their 
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costs. 
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Directorate,  US  Army  Concepts  Analysis  Agency  (CAA). 
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Analysis  Agency,  ATTN:  CSCA-FSR,  8120  Woodmont  Avenue,  Bethesda,  Maryland 
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